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The Arizona Head Start-Public School Transition Project was 
one of 31 federally created programs designed to improve low- income 
children's transition from Head Start to public schools through the third 
grade, making earlier interventions, and promoting continuity with the 
participation of parents, teachers, and community members. The Transition 
concept has promoted four primary family- centered services: developmentally 
appropriate education, health provision, family development, and parent 
involvement. Two consecutive cohorts of children entering kindergarten at 
different years were studied, with outcomes for children in the Transition 
treatment condition compared to those for children in a comparison condition. 
Analyses were conducted of outcomes for children, families, system and 
policy. Overall findings at the 5-year completion of the project indicate 
that the Transition Project produced few cognitive, social, or emotional 
results for Transition children, nor did it produce significant differences 
between Transition and Comparison families. The Project did influence 
systemic changes in educational practices and collaborative relationships . 
Based on findings, recommendations for agency and public policy include the 
following: programs should focus on attainable goals; accountability for 
results should be the cornerstone of new program designs and evaluations; 
future Health and Human Services (HHS) research should focus on determining 
the long-term effects of the Head Start Program, not solely on new 
demonstration projects; and new programs should align their goals with 
Welfare- to-Work reforms. (Three appendices include a summary of data 
collection instruments, a comparison of school practices and services as of 
1996, and a summary of results of longitudinal findings and analysis.) (JPB) 
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In 1 991 , a federal law created the Notionol Heod Stort-Public School Eorly Childhood 
Transition Demonstration Project. This program was designed to involve multiple sites 
across the country, with three objectives: First, each site would develop strategies in 
which Head Start, parents, local education agencies, and other community agencies would 
jointly and collaboratively plan and implement a comprehensive and continuous program of 
services for low-income children and their families, beginning in Head Start and continuing 
in the public schools through third grade. Second, each site would develop strategies to 
support parent involvement in their children's education. Third, results of the project 
would test the hypothesis that the benefits of Head Start could be maintained by 
extending the program's comprehensive services into kindergarten through third grade. 

Arizona was among 30 states that, along with the Navajo Nation, served as demonstration 
sites for the project. Southwest Human Development, Inc. (SWHD), a social service agency 
and Head Start grantee in Central Phoenix, was awarded the Arizona Project. SWHD 
contracted with the Morrison Institute for Public Policy at Arizona State University to 
conduct an independent evaluation. 

Each site evaluator collected the national core data set as specified by the national 
research coordinating team — Civitan International Research Center at the University of 
Alabama at Birmingham. Each site's national core data were collected from local 
participants (children, parents, teachers, principals, and family advocates). Because local 
evaluation plans reflected the goals of local Transition Projects, national data were 
supplemented by data derived from a combination of other published and 
locally-developed measures. 

During the course of the Arizona Project, which began in Fall 1992, three annual reports 
and one set of case studies were completed by the local evaluators. This final document 
presents the overall results obtained for the Arizona site of Head Start Transition at the 
completion of the Project. 

Traii§itioii CopQpoiieiit§ 

Planning for the Arizona Transition Project occurred at a time of growing consensus among experts 
that the educational and developmental needs of children would be better met by smoothing the 
transition from preschool to public school, making earlier rather than later interventions, and 
promoting continuity with the participation of parents, teachers, and community members. 

Program continuity would allow the families of young school age children to focus on nurturing 
and strengthening relationships rather than on repeatedly adapting to new educational and 
service systems. 
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The Transition concept is supported by four primary family-centered services: 
education, health, family development, and parent involvement. The components are 
described below: 

• Promoting and supporting education practices, curriculum, and materials 
that are developmentally appropriate: Developmental^ appropriate practices 
(DAP) are defined by the National Association for the Education of Young Children 
as "more active learning approaches based on a broader interpretation of children's 
educational needs and abilities rather than undue emphasis on rote learning and 
whole-group instruction of narrowly-defined academic skills." Among the strategies 
used in the Transition Project to promote DAP are the development of individual educa- 
tion plans for each child, the use of related classroom materials, and the opportunity 
for teachers to engage in intensive professional development. DAP, however, is not a 
curriculum, but a philosophy. 

• Providing-or linking children and families with-needed physical health, 
mental health, and dental services: Many educators favor linking health and 
social services to schools because academic achievement is likely to be impaired if stu- 
dents — or their parents — are chronically hungry, ill, or preoccupied with practical or 
personal problems. To assist Transition Project participants with such problems, "family 
advocates" (social service workers) were provided at each of the Transition schools. 

All of the advocates were bilingual. 

• Providing services to support and enhance family development: The most 
important conduits of services to families were the family advocates based at each 
Transition school site. Family advocates each worked with about 30 to 40 families to 
identify strengths and needs and to create individual action plans for addressing 
them. Family advocate activities included making home visits, providing parent 
training, offering referrals for services and educational programs, and providing 
direct services such as food or clothing. 

• Promoting parent involvement in their children's education, both at home 
and at school: Research documents a strong association between parental 
involvement and child academic achievement. The Transition Project made special 
attempts to extend parent participation in school at all levels: parents were 
encouraged to participate in advisory boards and volunteer in Head Start and 
elementary school classrooms; parent meetings and events were hosted by the 
schools; and teachers and family advocates visited children's homes. Advocates also 
translated for parents and school staff, teachers suggested learning activities for 
families to share at home, and advocates offered parenting classes as well as links 

to GED, ESL, and vocational classes. 
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Sammary of Re$alt§ 

The Transition Project utilized the quasi-experimental approach of the national study, 
which was designed to compare two separate groups composed of children and their 
families. Both groups had earlier participated in the Head Start program, but once in 
public school the treatment group received Transition Project services while the 
Comparison group did not. Two consecutive cohorts of children and families were studied: 
Cohort 1 children entered kindergarten in Fall 1 992; Cohort 2 children entered 
kindergarten in Fall 1993. 

The Arizona Project recruited students from a total of six schools — two in each of three 
central Phoenix elementary districts (Balsz, Creighton, and Osborn). Within each district, 
one of the schools was randomly designated the Transition site; the other the Comparison 
site. The Arizona Project provided services only to Transition School classrooms in which 
Transition participants were located. Data were gathered for all former Head Start 
children and families participating in the study. 

The Arizona Project's local evaluation plan was dependent upon the national design, but 
was customized to address issues of specific concern to the Arizona Project. The design was 
structured around a set of questions formulated through a content analysis of the project 
proposal, discussions with consortium members, and the Evaluation Advisory Panel. The 
resulting 12 evaluation questions were aligned with desired program outcomes and fell 
under the major categories of children, families, system, and policy. For this report, a 
longitudinal analysis was undertaken using data collected from both cohorts during the 
length of the Project. 

RESULTS FOR CHILDREN 

All of the evaluation questions regarding outcomes for children hypothesized that the 
services provided through the Transition Project would result in greater gains by children 
in the Transition treatment condition than by those in the Comparison condition. The data, 
however, provide evidence that this hypothesis is not accurate. Relatively few differences 
exist between Transition and Comparison group children; those that occur are generally 
small and tend to favor the Comparison group as often as Transition. 

On cognitive measures, longitudinal analysis reveals no differences between groups in 
reading performance by third grade. Final math scores show the Comparison group 
outperforming the Transition group in third grade, while the reverse was true at 
kindergarten. Vocabulary measures show the Transition group ahead of the Comparison 
group by third grade after starting out behind in kindergarten. The differences in both 
cases are small but significant. 






On measures of social and emotional development, Transition group children also do not 
exceed Comparison children. Children in both groups were similarly positive in their 
attitudes toward school. Longitudinal analysis of social skills measures and behavior 
problems according to parents show no differences between groups. By third grade, 
however, teachers who completed similar surveys reported that Transition students 
exhibited more externalizing problems (such as aggression) than Comparison students. 

In fact, the overall score for behavior problems was significantly higher for the Transition 
group than those in the Comparison group, indicating a greater number of problem 
behaviors overall. Parent ratings of their child's health showed no difference between 
Transition and Comparison groups. 

RESULTS FOR FAMILIES 

The Transition Project hypothesized that Transition services would positively impact 
families in a number of ways — from helping families obtain basic assistance, to improving 
self-sufficiency, parenting skills, and communications, relationships, and involvement with 
the schools. The family background component of the Family Interview, however, revealed 
no consistent differences between the groups over the course of the study regarding 
enrollment in social services. And while families reported less use of public services by 
their children's third grade year, no differences appeared between treatment groups. 

Families also reported higher incomes and greater employment, but again there were 
no differences between treatment groups. Analysis of family risk factors also revealed 
no differences. 

Regarding parent involvement in schools, results showed no differences in parenting skills 
between groups. In addition, there were no differences between groups in the types or 
frequency of communication and relationships with school staff, but Transition parents 
reported that their schools offered significantly more activities to parents than did 
Comparison parents. Similarly, Transition families were expected to have higher 
participation in literacy and English as a Second Language programs because of support 
made available to them, but the data revealed parallel participation in these activities. 

POSSIBLE EXPLANATIONS FOR RESULTS 

Given the lack of strong findings each year of the Transition Project, it is not surprising to 
find few strong longitudinal differences by treatment group. Indeed, similar results have 
occurred at a majority of the other 30 state sites participating in the national Transition 
Project evaluation. A few cogent factors might explain the lack of strong effects. These 
include contamination of treatment, attrition, English language proficiency, measurement 
problems, limited statistical power, and lack of data about family need and intensity of service. 
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OTHER ANALYSES CONDUCTED 



Because a number of confounding factors were identified as potential problems on the 
national core data set, local evaluators conducted additional analyses to determine whether 
previously untested factors might predict or explain some outcomes better than treatment 
group alone. This approach was also taken by other Transition Project site evaluations 
across the country. Analyses were conducted to understand the impact of attrition, family 
risk factors, and primary language. The role of other variables such as income, 
kindergarten test scores, and family involvement were also examined. Some findings did 
emerge, but given the extensive nature of the investigation, none were startling in their 
implications about children and families. 

SYSTEM AND POLICY RESULTS 

Qualitative data provide a picture of where and how systemic changes occurred, how 
collaborative partnerships worked, and what future attempts at collaboration can gain 
from this experience. From this data, substantial evidence indicates the Transition Project 
produced noticeable effects on both the public school system and the Head Start agency 
involved. While most of the data assessing systemic change were collected only at 
Transition schools, trends still can be gleaned from the data. Among the findings: 

In many cases, transition services became better coordinated for families. This happened 
when collaboration occurred among advocates, public school teachers, Head Start 
teachers, and social service agencies. Coordination was most apparent when continuity was 
maintained among all the parties involved. Transition teachers also gained understanding of 
the DAP approach and increased their DAP skills, primarily because the Project stood as a 
powerful model for effective teacher training. Moreover, the Transition Project effected 
widespread changes in the way teachers work with at-risk students, and it increased their 
sensitivity to diverse cultures. The former change was partially accomplished by training in 
DAP. Both changes, however, were largely expedited by the introduction into classrooms of 
bilingual and neighborhood-based advocates. Through the advocates' efforts to make 
teachers more comfortable with home visits and parents more comfortable in the 
classroom, an environment of mutual understanding between teachers and families 
developed. This led, in some cases, to improved parent involvement. Increased sensitivity to 
cultures also spread school-wide due to the Transition Project. 

Some of the changes may be short-term. Without advocates in the classrooms, home visits 
by teachers and other staff will diminish. And though some schools already have a social 
service worker on staff with some responsibilities similar to those of an advocate, the case 
loads these staff members carry are many times higher than those of advocates, so the 
intensity of services to families will undoubtedly suffer, as will coordination of services in 
individual classrooms. 











Other changes may be longer term. Because of the collaborative and partnering 
experiences that Transition has provided, lines of communication have been opened 
between school staff and other agencies, and a willingness has been fostered among some 
staff to consider alternative solutions to enigmatic problems. 

COLLABORATION 

A good deal was learned from the Project about working with other agencies toward a 
common goal. In addition, the Arizona Transition Project benefited from relationships with 
the Transition schools through existing Head Start programs at the schools. Still, much 
variation occurred among the three Transition schools. 

One of the primary factors that influenced success in working with other agencies was the 
readiness of the partners. One school entered the Project more ready than the others in 
terms of DAP training, existing links with social services, and Transition philosophy. Less 
time was needed by this school to surmount barriers and move forward with collaborative 
efforts. The less-ready Transition schools also developed collaborative relationships, but 
their efforts required more communication and patience. 

Among the most important barriers to collaboration for the Transition Project were staff 
turnover among family advocates — a situation that creates the potential for breakdown in 
communication between groups; expansion of the Transition Project to new teachers every 
year — a factor that served to forestall significant buildup of momentum for Transition; and 
knowledge that the Project would lose its funding at the end of five years. Given these 
factors, the Transition Project collaboration was probably as successful as it could be. 

Ultimately, both Southwest Human Development and the public schools involved became 
more aware of the benefits and pitfalls of collaborative endeavors as a result of the 
Transition Project. SWHD is currently applying its experience to a state-funded project 
that requires collaboration with other community-based service providers in developing a 
new child care center for low-income working families. 

As a result of successful collaboration some program "blurring" also occurred, and the 
services of Head Start and Head Start Transition became indistinguishable to the 
beneficiaries. This should be considered a symptom of success. 

EFFECTS OF TRANSITION ON PUBLIC POLICIES 

The final goal of the Transition Project was to impact state policy with the results of the 
Project. At this date, there is no evidence that this has occurred, but given how public 
policy is set, it may be unreasonable to expect one project's results to be sufficiently 
powerful to stimulate any changes in the short term. 








A better way to affect state policy may be through a coalition of organizations with 
similar goals. Transition Project personnel are currently assisting two such coalitions 
by applying the experiences they gained from the Project. 



In summary, the Transition Project produced few cognitive, social, or emotional results for 
Transition children, nor did it produce significant differences between Transition and 
Comparison families. The Project did, however, influence systemic changes in educational 
practices and in collaborative relationships. The public schools' experiences with 
collaboration may prove to be the greatest long-term benefit of the Project. 



A number of recommendations are offered. Some recommendations relate to internal 
(agency policy), others relate to public policy. Both are shown here: 



INTERNAL POLICY (Service Delivery) RECOMMENDATIONS: 

• SWHD managers should revisit hiring, training, and supervising practices to determine 
whether they are sufficient to support their advocates' work. Some advocates 
under-utilized or were not uniformly familiar with some services, specifically those 
related to employability and employment. 

• Program managers should determine what is a working definition of an effective 
family advocate, what are expectations for advocates in responding to clients and 
supervisors, and what are the consequences for advocates who do not prove 
effective. These steps will help satisfy increased demands for program accountability. 

• Future program managers should determine which parents are most likely to participate 
at a high level, and then keep records of service provision and intensity to uncover 
reasons behind results. This will help programs make the best use of available services. 

• Program managers should continue to seek better service continuity for families and 
teachers. One positive strategy may be looping," which keeps a teacher and 
advocate with the same set of students (and families) through more than one grade 
level before looping them back to an earlier grade to pick up a new set of students. 

• Managers should reconsider how advocates allocate time. Although paperwork and meetings 
are essential, an effective balance must be struck between time spent on administrative 
tasks and time spent providing direct service. The goal should be to maximize staff contact 
with families. 
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PUBLIC POLICY RECOMMENDATIONS 

• RFP evaluation processes and standards should be reformed to favor 
programs that provide intensive services. 

• Program designs should focus on attainable goals. 

• Accountability for results should be the cornerstone of new program 
designs and evaluation. 

• The value of large-scale demonstration projects must be weighed 
against the expense and associated threats to validity. 

• Future HHS research should focus on determining the long-term effects of the 
Head Start Program, not solely on new demonstration projects. 

• New programs should align their goals with Welfare-to-Work reforms. 
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...FROM TRANSITION PROJECT EVALUATION 
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Evaluation Question Finding 



Children 

1. Do Head Start (HS) children in Transition classrooms maintain 
and/or show gains to a greater degree than HS children in 
Comparison classrooms on the following indicators: 

a) cognitive skills 

b) social and emotional development 

c) general health 

d) adjustment to school? 

2. Do HS children in Transition classrooms exhibit more positive attitudes 
toward school than HS children in Comparison classrooms? 

3. Do HS children in Transition classrooms experience a smoother 
transition and better continuity of programming from HS to 
kindergarten and from one primary grade to the next than HS 
children in Comparison classrooms?* 

Families 

4. Do Transition families receive more social service support through the 
public school system and show more evidence of stability and 
self-sufficiency than Comparison families? 

5. Do Transition families show better parenting skills and have more 
involvement in and support for education than Comparison families? 

6. Do parents in Transition schools participate in and complete more 
literacy and English as a Second Language classes and workshops than 
parents in Comparison schools? 



a) differences on standardized measures for math favor 
Comparison; differences on vocabulary favor Transition; 
no significant differences on reading 

b) few differences favor Comparison 

c) no significant differences 

d) no significant differences 

no significant differences 

evidence of more early childhood programming and 
focus on continuity at Transition schools 



evidence of more services; no significant differences on 
stability or self-sufficiency 

no significant differences 

no significant differences 
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Sammary of Findings... 









...FROM TRANSITION PROJECT EVALUATION 




Evaluation Question 



Finding 



7. Do parents in Transition schools perceive home-school 
communication to be more effective and satisfactory than 
parents in Comparison schools? 

System 

8. Do Transition schools provide a more coordinated service 
delivery system (i.e., continuous and comprehensive) than 
Comparison schools?* 

9. Do Transition schools provide a more developmentally appropriate 
curriculum, more satisfactory communication strategies, better 
staff development, and more opportunities for parent participation 
than Comparison schools? 

10. Are Transition school primary level teachers more skilled in 
working with the special needs of at-risk children and families 
than Comparison school teachers?* 

11. What does a successful collaborative process look like?* 



no significant differences 



differences favor Transition schools 



differences on Developmentally Appropriate Practice 
favor Transition; no significant differences on 
communications; some differences on opportunities 
for parent participation favor Transition 

evidence of cultural and linguistic sensitivity in 
planning and implementation at Transition schools 



evidence of collaboration; barriers identified and 
collaborative strategies learned; positive experience 
promotes future collaboration 



Policy 

12. Have the results of this Project affected state and local level 
policies and level of fiscal support that reflect a comprehensive 
plan for addressing child and family needs in a holistic manner? 



not at completion of Project; however lessons 
learned from Transition Project will be applied to 
coalitions supporting child and family issues 
which may affect future policy 



*These questions rely on multiple data sources. Direct comparisons are not always possible as some data were not available for Comparison group schools. 
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Iiltrodactioii 

One child in five in the United States lives in poverty, a circumstance that places 
this child at a serious educational disadvantage. Public schools have sought to 
mitigate the problem by equalizing opportunities for children regardless of 
social and economic factors. While these efforts have proven successful for 
millions of children and young adults, the fact remains that too many students 
drop out or leave school without the skills necessary to compete successfully in 
the market place — and their numbers are disproportionately high among poor 
and minority groups. 

Head Start was created in 1 965 as a means of breaking the cycle of poverty for 
young children. Early supporters believed that education was the key to achieving 
this goal, therefore the first Head Start programs typically centered on summer 
preschools that were supplemented by medical, dental, mental health, and social 
services. With early exposure to educational opportunity and social services, the 
disadvantaged children who participated in these programs were expected to enter 
public school "ready to learn." 

Although Head Start began as a subdivision of the Office of Economic 
Opportunity, it was transferred to the Department of Health and Human Services 
(HHS) in the early 1970s where today it is overseen by the Administration on 
Children, Youth and Families (ACYF). The move to ACYF reflected increased 
emphasis on family and community involvement as well as a commitment to 
developing innovative ways for improving Head Start's effectiveness. 

More than 1 5 million children have participated in Head Start during the past 
three decades at a total cost of approximately $31 billion. 1 Currently, 830,000 
students attend Head Start — almost all enrolled for nine months. While the Head 
Start program has undergone numerous changes in its thirty-plus years, public 
regard for the program has remained high. Nevertheless, some controversy over 
the program exists. Some politicians want to increase funding in order to serve all 
eligible children every year (approximately two million); others suggest funding 
cuts because of questions about the program's effectiveness. 





■tr *0 t StrK ~- 



ln an attempt to understand Head Start's impact, HHS commissioned a 
comprehensive literature review to analyze existing research on the program. The 
authors of the resulting meta-analysis considered a total of 210 research articles 
published between Head Start's inception and 1985. 2 They concluded that Head 
Start students usually showed gains in intelligence, achievement, and social 
behavior and attitudes during the preschool program, but these gains diminished 
when measured one to three years later — an effect commonly referred to as the 
"fade out effect." And though a few studies indicated that Head Start graduates 
were less likely than peers to be retained in a grade or placed in special education 
classes, the authors determined that to more confidently assess Head Start 
outcomes, large-scale longitudinal evaluations were needed. 

The evidence of a fade-out of benefits and the call for more research were 
important motivational factors that preceded the passage of the 1991 federal law 
creating the National Head Start-Public School Early Childhood Transition 
Demonstration Project. This new program was designed to involve multiple sites 
across the country, with objectives (as stated in the July 11, 1991 Federal 
Register) to be threefold. First, each site would develop strategies in which Head 
Start, parents, local education agencies (LEAs), and other community agencies 
would jointly and collaboratively plan and implement a comprehensive and continuous 
program of services for low-income children and their families, beginning in Head 
Start and continuing in the public schools through third grade. Second, each site 
would develop strategies to support parents involvement in their children's 
education. Third, results of the project would test the hypothesis that the 
benefits of Head Start could be maintained by extending the program s 
comprehensive services into kindergarten through third grade. 3 

Arizona was among 30 states that, along with the Navajo Nation, served as 
demonstration sites for the project. Southwest Human Development (SWHD), 
a social service agency and Head Start grantee in Central Phoenix, was awarded 
the Arizona Project. As a requirement of participation, each Transition site 
hired an independent evaluator to conduct a local evaluation. For its evaluation, 

SWHD subcontracted with the Morrison Institute for Public Policy at 
Arizona State University. 

As a condition of involvement, each site evaluator was required to collect the 
national core data set as specified by the national research coordinating 
team — Civitan International Research Center at the University of Alabama at 
Birmingham. Each of the 31 Transition Project sites and evaluators also 
participated with the national research coordinating team in the National 
Transition Demonstration Consortium, whose goal was to facilitate planning and 
communication regarding the Project and the national evaluation. 
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Each site's national core data were collected from local participants (children, parents, 
teachers, principals, and family advocates) every year and then sent to Civitan where it was 
analyzed and returned to the sites for use in the local evaluations. Because local evaluation 
plans reflected the goals of local Transition Projects, national data were supplemented by 
data derived from a combination of other published and locally-developed measures. 

During the course of the Arizona Project, which began in Fall 1992, three annual reports 
and one set of case studies were completed by the local evaluators. 4 This final document 
presents the overall results obtained for the Arizona site of Head Start Transition at the 
completion of the Project, and is broken down as follows: Chapter 1 provides a brief history 
of the Transition Project, and includes descriptions of the major program components and 
the quasi-experimental designs and methods for both the national and local evaluations. 
Chapter 2 summarizes results for the Arizona site based on analysis of data according to 
the national evaluation plan, plus other analyses. Chapter 3 presents results related to the 
systemic change and collaboration goals of the Transition Project , which were the major 
focus of the local evaluation plan. Chapter 4 discusses results and examines public policy 
implications of the Project based on local findings. 

Traiisitioii Components, Laws 
and Related Research 

A diverse body of research provides strong argument for smoothing transitions 
between Head Start and public schools. This section describes the rationale behind 
the transition concept, presents the legislative context, and discusses the 
Transition components and relevant research. 

SUSTAINING CONTINUITY 

Planning for the Arizona Transition Project occurred at a time of growing 
consensus among experts that the educational and developmental needs of children 
would be better met by smoothing the transition from preschool to public school 
and by making earlier rather than later interventions. Today, many experts also 
agree that the early childhood years should be treated as a continuum, and that 
early programs and services should be sustained with the consistent participation 
of parents, teachers, and community members. Program continuity, it is argued, 
allows the families of young school age children to focus on nurturing and 
strengthening their relationships rather than on repeatedly adapting to new 
educational and service systems. The "guiding principles" for continuous effective 
interventions include high quality, comprehensiveness (both of services and family 
involvement), and extension over time. 5 












Historically, however, public schools have not embraced the idea of continuity and smooth 
transitions, even for children entering kindergarten. The US Department of Education 
surveyed nationally representative samples of 830 school districts and 1,1 69 schools to 
assess the incidence of transition activities between preschools and kindergarten, and found 
that, of those surveyed, only 10 percent reported systematic communication between 
kindergarten teachers and previous teachers or caretakers, only about 12 percent of schools 
said they coordinated curricula, and fewer than half hosted a formal visitation program for 
parents of incoming children. In addition, "high levels of child difficulty adjusting to the 
academic demands of kindergarten" were reported by 33 percent of schools in which the 
majority of children were poor, as compared to 6 percent of schools in which less than a 
quarter of students were poor. 6 Given these circumstances, Transition sites were intended to 
foster the following conditions: 

• direct communication among teachers between and within institutions 

• transfer of children's records with developmental information 

• joint in-service training for teachers and other staff members 

• coordination of curricula for continuity among programs 

• activities to prepare children and parents for transitions 

• advance visits to new classrooms for children and parents 

• administrative cooperation and coordination 

These form the essence of the transition concept. At the national level, efforts to smooth 
transitions for children and families between systems and from grade to grade in the public 
schools were to be considered effective if the following resulted: 

• Children had positive attitudes toward school and were motivated to do well in school. 

• Parents and other key adults in young children's lives displayed positive attitudes 
toward their children's school program and were active partners (along with school 
personnel) in children's learning. 

• Teachers and principals recognized individual differences in children's 
academic and social-emotional maturity, positively valued cultural and 
linguistic diversity, and provided developmental^ appropriate experiences 
within the class and school setting. 

• Positive and mutually supportive relationships occurred among families, 
school personnel, social and health service providers, and communities 
concerning the well-being and education of young children. 7 

To facilitate successful transitions at each Transition Project site, representatives of the 
parents, the schools, and the local communities served on local governing boards where they 
guided program design, implementation, and oversight. These representatives ensured that 
services were appropriate for diverse ethnic groups, for those with disabilities, and for the 
various participating organizations. 















FEDERAL AND STATE LEGISLATIVE CONTEXT 

Support for Head Start has been evident at both the Federal and state level. In 1994, 
the U.S. Congress re-authorized both the Head Start Act and the Elementary and 
Secondary Education Act (ESEA), reinforcing support for the Transition concept of early 
childhood education. In July 1996, the ACYF announced that an additional $35 million 
would be made available to Head Start grantees as increases beyond their ongoing Head 
Start grants, beginning with programs re-funded on July 1, 1996 and continuing with 
monthly re-funding through June 1,1997. 

In November 1996, the first comprehensive revision in 20 years of Head Start 
performance standards was published. In some respects the new standards were 
stricter, for example cutting the time period in half for completing screenings of 
each child's developmental, sensory, and behavioral levels. In other cases the new 
standards bowed to local laws, for example deferring to state or municipal 
standards regarding tuberculosis screenings for volunteer Head Start workers. 

More recent national-level government pronouncements bode well for the 
Transition concept and other innovative educational programs. President Clinton, in 
his State of the Union address of January 1997, announced that improving the 
nation's educational system would be the central theme of his second administration. 
Mentioning the Head Start Program by name, he set explicit goals for the year 2000, 
stating that every child should read by the age of eight and have access to the 
Internet by age 12. 

The 1998 Head Start reauthorization was signed by President Clinton on October 
21. With an increased budget of $312 million targeted toward expansion and 
improvements in quality, research and specific programs such as Migrant and Early 
Head Start, congressional and presidential support for Head Start remain steady. 

On the state level, Arizona has demonstrated its own commitment to transition 
practices. In March 1996, the Arizona Head Start Collaboration Project was 
initiated by the Arizona State Head Start Association, a representative body 
composed of Head Start directors and parents from Southwest Human 
Development and six other regional Head Start grantees and tribal agencies in 
the state. Under the Arizona Governor's Division for Children, the Head Start 
Collaboration Project, which was made possible with funding from the HHS, has 
created a formal partnership of state government agencies, Head Start programs, 
and other interested parties as an infrastructure for coordinating policy, planning, 
and service delivery in the areas of health, child care, welfare, education, literacy, 
and activities related to children with disabilities. 
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Recently-enacted welfare reform legislation on both the national and state levels, 
however, will impact services to vulnerable families. The "Personal Responsibility 
and Work Opportunity Reconciliation Act of 1996" ended federal guarantees of 
assistance to the poor and created capped block grants to states called Temporary 
Assistance for Needy Families (TANF). Arizona's EMPOWER (Employing and 
Moving people Off Welfare and Encouraging Responsibility) Program, which took 
effect in November 1995 and runs through 2002, is less restrictive than TANF in 
certain respects. A Block Grant Committee was established in the state House of 
Representatives to shape policies and legislation, and to explore how much 
flexibility is permitted individual states under the TANF block grant. 

In 1998, two pieces of legislation supporting low income and at-risk families were 
passed. Kids Care, a bill to provide health care to the children of the working poor, 
and Healthy Families, a child abuse prevention program that identifies families at 
great risk of abuse and intervenes with an intensive home visiting program. 
Although conditions for children, overall, are considered poor in Arizona, 8 and 
legislation supportive of children and family issues within the last ten years have 
not been prevalent, the 1998 laws represent increased support for struggling 
families. 

COMPONENTS OF THE TRANSITION PROJECT 

The Transition concept is supported by four primary family-centered services: 
education, health, family development, and parent involvement. While some of 
these services were offered as a matter of policy at Transition and Comparison 
schools prior to the Transition Project, during the Project all four services were 
provided at all three Transition schools, with certain variations dictated by the 
needs and preferences of individual schools. Examples for each component follow. 



W&at is De-velopmeiitally 
Appropriate Practice? 

D evelopmentally appropriate practices (DAP) 
are defined by the National Association for the 
Education of Young Children (NAEYC) as "more 
active learning approaches based on a broader 
interpretation of children's educational needs and 
abilities rather than undue emphasis on rote 
learning and whole-group instruction of narrow- 
ly-defined academic skills ." 9 DAP is not a cur- 
riculum, but a philosophy based upon child 
development research and learning theory. DAP 
advocates believe that young children are 
intrinsically motivated to learn, and that edu- 
cational practices are most effective if they 
support a child's natural tendency to explore 
and experiment. 

DAP differs from other approaches to teaching 
in a number of ways. The child is not viewed 
simply as the recipient of knowledge transmit- 
ted by others. Instead the child actively con- 
structs knowledge through direct experience 
with the environment. Instructional decision 
making in the DAP classroom is based upon 
knowledge of child development and what is 
age appropriate for children as well as what is 
individually appropriate for a particular child. 
The role defined for teachers is also different. 
Traditional educators typically choose learn- 
ing experiences for their students and rely 
heavily on formal instruction. DAP-oriented 
teachers, however, often act as facilitators, 
creating a range of opportunities that allow 
children to direct their own learning. While 
most teachers would fall somewhere between 
completely traditional or completely DAP-ori- 
ented, the following vignettes help illustrate 
differences between traditional and develop- 
mentally appropriate teaching styles by con- 
trasting some of the attributes commonly asso- 
ciated with each approach. 

• The traditional classroom is quiet and order- 
ly, consistent with school rules posted on the 
bulletin board. Supplies are stored in closed 
cabinets to be used at the teacher's discretion. 
Students sit in rows at individual desks facing 
the teacher and blackboard. 



This morning they are finishing worksheets that complement the basal readers they use in their assigned, ability-based reading groups. 
They occasionally raise their hands if they need the teacher's help, but will continue working until the teacher tells them reading peri- 
od is over. Next they will rehearse their multiplication tables as a group, then the teacher will present a social studies lesson to the 
whole class. 



In the developmentally appropriate classroom, pupils proposed and voted on the rules posted in the room. There is often a high 
level of activity. While small groups of students gather around learning centers and work with equipment, others talk among them- 
selves as they solve problems structured by the teacher. Materials for children's use are grouped on low, open shelves and students 
use them during designated activities. The teacher continually moves among groups, asking questions or making suggestions to facil- 
itate the children's learning, later the teacher may have the students write about their experiences or research a related topic. 
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• Promoting and supporting education practices, curriculum, and 
materials that are developmental^ appropriate. 

While DAP is a common approach among preschool practitioners, and support is growing in 
the primary grades, some differences of opinion exist regarding its use in the primary level 
school setting. Research, for example, indicates that traditional academic environments 
may result in higher academic achievement as measured by standardized tests. Emotional 
costs, however, may accompany this higher achievement , 10 and some research indicates that 
traditional instruction alone is not necessary for academic success. DAP's beneficial effects, 
according to its advocates, include less stress, greater creativity, more self-confidence in 
cognitive skills, and sustained motivation and interest in school. The Transition Project 
encouraged use of DAP in the classroom, and also sponsored intensive professional 
development for teachers, equipment purchases related to DAP, and parent training. 

The three Transition schools implemented the DAP component in a number of ways. 

Encanto, for example, provided well-organized DAP trainings for teachers, supplied 
a wide variety of literature and other materials for classrooms, and also placed special 
emphasis on arranging for teachers to visit classrooms that featured multi-age and bilingual 
programs, the use portfolios for assessment, and other developmental^ appropriate 
approaches. Machan held workshops and seminars on bilingual literacy, problem-solving, 
performance assessments, and the Project Approach to curriculum planning, and also made 
funds available for the purchase of computers, furniture, and other equipment and supplies. 
Crockett also offered workshops, the assistance of consultants in classrooms, and time off for 
teachers to visit other classes. In addition, some Crockett teachers began the practice of 
"looping," which keeps them with the same class of students for two consecutive years. 

• Providing-or linking children and families with-needed physical 
health, mental health, and dental services. 

Many educators favor linking health and social services to schools because academic 
achievement is likely to be impaired if students — ar their parents — are chronically 
hungry, ill, or preoccupied with practical or personal problems. To assist Transition Project 
participants with such problems, "family advocates" (social service workers) were provided 
at each of the Transition schools. All of the advocates were bilingual. 

The health delivery system for the Transition Project was modeled after that used by 
SWHD's Head Start program. It offered treatment and referrals related to physical, 
mental, and dental health needs of participating families. A nurse coordinated health 
services with school nurses and consulted with family advocates and families about health 
issues. The Project often provided major dental care through cooperating dentists when 
needed. As the pool of recipients grew over time, all three Transition schools worked to 
make this component more comprehensive and efficient. Among the steps taken were 
increased efforts to ensure that immunizations, vision and hearing screenings, fluoride 
treatments, and other health and dental services were provided on a timely basis. 













• Providing services to support and enhance family development. 

The most important conduits of services to families were the family advocates 
based at each Transition school site. Family advocates each worked with about 30 
to 40 families to identify strengths and needs and to create individual action plans 
for addressing them. Family advocate activities included making home visits, 
providing parent training, offering referrals for services and educational 
programs, and providing direct services such as food or clothing. 

As the Transition Project moved into its middle years and the parameters of its 
family service mission were repeatedly stretched and redefined, each advocate's 
flexibility and creativity was continually challenged. Among other things, advocates 
represented a bridge between cultures, often serving as translators, enrolling 
family members in ESL classes or encouraging classroom teachers to learn 
rudimentary Spanish or enlist Spanish-speaking aides. 

• Promoting parent involvement in their children's education, 
both at home and at school. 

Research has consistently documented a strong association between parental 
involvement and children's academic achievement. The Transition Project, therefore, 
made special attempts to support and extend parent participation in school at all levels. 
Several strategies were employed: parents were encouraged to serve on advisory boards 
and volunteer in both Head Start and elementary school classrooms; special parent 
meetings and events were hosted by the schools; and before the school year began, 
teachers and family advocates visited children's homes. 

Advocates also provided translation services for parents and school staff. Teachers 
suggested learning activities for families to share at home and ways to enhance their 
children's learning. Family advocates and community organizations encouraged parents 
to model learning, and also offered parenting skills classes as well as links to GED, ESL, 
and vocational classes. 

A variety of activities were offered to attract parents to the school and involve them in the 
education of their children. At Machan School, for example, a Thanksgiving potluck meal 
was scheduled to be held each year at lunchtime in order to allow working parents to share 
the meal with their children. 
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E-valaatioii Design 

The Arizona Head Start - Public School Transition Project utilized the quasi-experimental 
approach of the national study, which was designed to compare two separate groups 
composed of children and their families. Both groups had earlier participated in the Head 
Start program, but once in public school the treatment group (called the Transition group) 
received Transition Project services while the Comparison group did not. In accordance 
with the national study design, two consecutive cohorts of children and families were 
studied: Cohort 1 children entered kindergarten in Fall 1992; Cohort 2 children entered 
kindergarten in Fall 1993. 

The National Head Start-Public School Transition Demonstration Project gave each 
local Transition Project grantee the latitude to design a unique service delivery 
model and determine the number of schools, school districts, and students 
involved. The Arizona Project elected to recruit students from a total of six 
schools — two schools in each of three central Phoenix elementary districts. Within 
each district, one of the schools was randomly designated the Transition site; the 
other the Comparison site. The districts and schools, by treatment condition are 
shown below. At the school level, the Arizona Project, like all other Transition Project 
sites, provided services to each classroom in which Transition participants were located. 
Data, however, were only gathered for former Head Start children and families 
participating in the study. 



Transition 

Balsz Elementary District: Crockett School 

Creighton Elementary District: Machan School 

Osborn Elementary District: Encanto School 



Comparison 
Balsz School 
Papago School 
Longview School 



The Arizona Project's local evaluation plan was dependent upon the national design 
and included the requirement to collect information for the national core data set. 
The local plan, however, was customized to address issues of specific concern to 
the Arizona Project, and was structured around a set of questions formulated 
through a content analysis of the project proposal, discussions with consortium 
members, and review and critique by an Evaluation Advisory Panel consisting of 
researchers from the participating districts. The resulting 12 evaluation questions, 
listed below, are aligned with desired program outcomes and fall under the major 
categories of children, families, system, and policy. 






Children 

1 . Do Head Start (HS) children in Transition classrooms maintain and/or show 
gains to a greater degree than children in Comparison classrooms on the 
following indicators: cognitive skills; social skills; emotional development; 
general health; and positive adjustment to the public school setting? 

2. Do HS children in Transition classrooms exhibit more positive attitudes toward 
school than children in Comparison classrooms? 

3. Do HS children in Transition classrooms experience a smoother transition and 
better continuity of programming from HS to kindergarten and from one primary 
grade to the next than HS children in Comparison classrooms? 

Families 

4. Do Transition families receive more social service support through the public school 
system and show more evidence of stability and self-sufficiency than Comparison families? 

5. Do Transition parents show better parenting skills and have more involvement and 
support for education than Comparison families? 

6. Do parents in Transition schools participate in and complete more literacy and English 
as a Second Language classes and workshops than parents in Comparison classrooms? 

7. Do parents in Transition schools perceive home-school communication to be 
more effective and satisfactory than parents in Comparison schools? 

System 

8. Do Transition schools provide a more coordinated service delivery system 
(i.e., continuous and comprehensive) than Comparison schools? 

9. Do Transition schools provide a more developmental^ appropriate curriculum, 
more satisfactory communication strategies, better staff development, and more 
opportunities for parent participation than Comparison schools? 

10. Are Transition school primary level teachers more skilled at working with the 
special needs of at-risk children and families than Comparison school teachers? 

11. What does a successful collaborative process look like? 

Policy 

1 2. Did the results of this project affect state and local level public policies and 
the level of fiscal support that reflect a comprehensive plan for addressing 
child and family needs in a holistic manner? 




fetameiitS a*ld Data Collection 



The local evaluation relied on two categories of data collection and analysis: 1 ) the 
national core data set that is included in the national Head Start Transition study, 
and 2) data on other matters of specific interest to the Arizona Project. These 
data, both quantitative and qualitative, were intended to determine the effects of 
Transition services by comparing attributes of the Transition group with those of 
the Comparison group. 

While the national data set primarily gathered quantitative information about 
children and families, the local data set focused on more qualitative data regarding 
the Project's impacts on schools and the Head Start program. Because the local 
evaluation's purpose was to learn more about the Project's systemic effects, much 
of the local data were collected only at Transition schools. 

During the course of the study, five separate data collection cycles occurred for 
each cohort. Baseline data were collected during the fall of each cohort's 
kindergarten year. Most instruments were then re-administered every spring 
through each cohort's third grade year. Table 1.1 shows the wide range of data, 
both national and local, that were collected and analyzed during the study. Unless 
otherwise noted, instruments shown were administered during the fall of each 
cohort's kindergarten year, and the following four spring data collection cycles. It 
should be noted that this report does not profile all of the results generated by 
each instrument administered for the national data set, since some were never 
relevant to local evaluation questions. Descriptions of each of the instruments for 
which data are reported appear in Appendix A. 

For the purposes of evaluation, all participants in the Transition Project were 
considered sources of data: children, families, teachers, family advocates, and 
principals. National core data and teacher ratings of students were collected from 
participants in both Transition and Comparison groups. Other local data related 
exclusively to the implementation of the Transition Project and were collected only 
from Transition school participants. A review of the data collection protocols follows. 
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Child assessments of cognitive development were administered to children individually in 
school settings outside the classroom. The assessments were conducted by examiners trained 
specifically in their use. Six instruments were used: the Peabody Picture Vocabulary 
Test-Revised (PPVT-Rf, the Spanish language adaptation of the PPVT-R, the Test De 
Vocabulario en Imagenes Peabody (TVIP); and four subtests of the Woodcock-Johnson 
Achievement Tests-Revised (WJ-22, letter-word recognition; WJ-23, passage comprehension; 
WJ-24, calculation; WJ-25, problem solving). 

Table 1.1 



DATA ! 


SETS ADMINISTERED AND ANALYZED 


Child Data 


NATIONAL 


Peabody Picture Vocabulary Test - Revised (PPVT-R) 1 

Test de Vocabulario en Imagenes Peabody (TVIP) tf 

Woodcock Johnson - Revised subtests 22, 23, 24, 25 

Writing Samples 3 

What 1 think of School 

Child's Adjustment to School - Parent 

Social Skills Rating System - Parent and Teacher Versions 

Child Health - Parent 




LOCAL 


Teacher Rating of Students 


Family Data 


NATIONAL 


Family Involvement in Children's Learning 3 
Parenting Dimensions 
Family Resource Scale 
Final Interview for Families 4 




LOCAL 


Local Family Interview Questions 
End-of-Year Summary of Family Services 


System Data 


NATIONAL 


Assessment Profile 
ADAPT 3 

Program Implementation Profile (PIP) - 1996 
School Climate Survey - Parent, teacher, principal 
Annual Site Visit Reports 




LOCAL 


Innovation Component Checklist - Transition Services 
Innovation Component Checklist - DAP 
Focus Groups 

Interviews with Principals - 1 994, 1 996 
Observations/documentation of program activities 
Survey of Collaboration 



1 Administered to each cohort in kindergarten and grodes 1 ond 3 

2 Spanish version administered in kindergarten on recommendation of teocher along with English, until teacher judged that child's 
English proficiency was adequate 

3 Administered to each cohort in grodes 2 ond 3 

4 Administered to each cohorts in Spring of child's third grode yeor 
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Children whose primary language was English were tested with each of the 
English-language achievement instruments. Children whose primary language was 
Spanish were given only the TVIP and the PPVT until they were deemed ready by 
their teachers to take the Woodcock-Johnson tests in English. The 
Spanish-language tests were administered by a bilingual examiner. 

FAMILY INTERVIEWS 

Family interviews were conducted at baseline and each spring thereafter. 

Interviewers who conducted family interviews were trained by a member of the 
local evaluation team who had previously been trained by the National Research 
Coordinating Team. Interviewers used machine-scannable interview booklets 
included in the national core data set. Spanish-speaking parents were interviewed 
using the Spanish-language version of the interview, while bilingual parents were 
given the choice of Spanish or English. 

All interviews were conducted at a time and place convenient to the family. Most 
parents preferred to be interviewed in their homes. Typically, the interviews took 
45 to 60 minutes to complete, and parents who participated received a $20 money 
order and a note of thanks. 

In some cases, families were not interviewed either because they declined, repeatedly 
canceled, or failed to keep interview appointments, or because they could not be reached 
after repeated attempts at contact. Thus, the possibility exists that interview data could be 
skewed by the "self-selection" of cooperating families. 

SURVEYS 

Surveys were completed by principals, teachers, and family advocates. Packages containing 
the survey instruments, instructions, and return envelopes were mailed out each spring 
with the request that they be returned within one month. Surveys not returned by that 
time were solicited by follow-up phone calls and/or personal notes. 
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CLASSROOM OBSERVATIONS 

Observations were made in the classrooms of participating teachers each spring. 

During the middle of the study (1994), the National Research Coordinating Team 
suggested that observations be limited to classrooms in which at least two children 
were participants of the study. This suggestion was incorporated by the local 
evaluation. All observations were conducted by a trained member of the evaluation 
team, and were scheduled at the individual teacher's convenience. Two instruments 
were used: the Assessment Profile for Early Childhood Programs, which was used 
from 1993 to 1997, and A Developmental^ Appropriate Practice Template 
(ADAPT), which was introduced in 1995 and used for the remainder of the study. 

INTERVIEWS AND OBSERVATIONS 

Focus group meetings were held each spring to elicit qualitative information regarding the 
perspectives of various stakeholders in the Transition Project. The focus group methodology 
described by Stewart and Shamdasani (1990) was used in recruiting participants, designing 
questions, conducting the dialogue, and analyzing the resulting data. A member of the 
evaluation team acted as facilitator for the sessions, which lasted about two hours and 
were audio taped for later transcription. Interview protocols were developed by the 
evaluator with input from members of the Transition management team. Sessions were 
held with the following groups: 

• Spanish-speaking and English-speaking parents of children enrolled in Transition 
classrooms from both cohorts and all three Transition schools 

• Transition classroom teachers from Transition schools 

• Transition family advocates and Head Start family advocates from Transition schools 

Transition school principals were interviewed individually in 1994 and 1996. 

Interview questions covered family services, health services, parent involvement, 
and teacher training in developmentally appropriate practice. Comparison school 
principals were interviewed in spring of 1994 and 1996 to document existing 
programs and services at Comparison schools that most closely resembled 
Transition Project services. A list of those Transition-like services, as of 1996, 
is shown in Appendix B. 
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The lead evaluator attended key Transition Project meetings and events throughout the 
Project, usually in the role of a participant/observer. The evaluator also attended most 
monthly meetings of the Transition Team at each of the three Transition schools as well as 
the monthly meetings of the Transition Governing Board, and participated in Transition 
Management meetings with Southwest Head Start managers held approximately every six 
weeks. The evaluator also attended a variety of professional development workshops and 
conferences related to the Project. Minutes of all meetings and notes taken by the evalua- 
tor/observer were logged and analyzed as part of the qualitative data set. In addition, a 
review of the family advocate end-of-year summary sheets was conducted to document ser 
vices to families in the Project. 

participants 

Participants, for the purpose of this study, were former Head Start children and their 
families who, in Fall 1992 or Fall 1993, enrolled for kindergarten in one of the three 
Transition schools or one of the three Comparison schools. Participants, therefore, included 
both those who received Project services (Transition school participants), and those who did 
not (Comparison school participants). Furthermore, as teachers received Project services, 
they also were considered participants. Total participant numbers for whom data are avail- 
able during the Project are shown in Table 1.2. 



Table 1.2: Participant Counts for Each Data Collection Cycle, with Cohorts Combined 




Families 

Interviewed 


m 


92 




108 


76 


10 


92 


65 


10 


71 


56 


37 


64 


54 


80 


Children 

Tested 


137 


110 




124 


99 




97 


76 


14 


72 


59 


44 


65 


54 


83 


Teachers 

Surveyed 








9 


13 




15 


25 




16 


27 




13 


19 








Cohort 1 participant families were described demographically in the 1992-93 report of the 
Transition Project, while Cohort 2 participants were described in the 1993-94 report. Unlike 
Cohort 1, which showed nearly identical demographic profiles for its Transition and 
Comparison families, Cohort 2 showed major differences between groups. 

The Transition group contained more than twice as many Hispanic families (82 percent) as 
the Comparison group; were twice as likely to speak Spanish at home (49 percent); 
included more respondents born outside the United States (mostly from Mexico); were 
more mobile; and had a lower median annual household income. The only statistically 
significant difference, however, was language spoken, an issue that is most relevant to the 
interpretation of child outcomes, and which has been taken into account statistically. 
Therefore, to increase the statistical power of the longitudinal analysis, the two cohorts 
were combined. 



Analysis 

Statistical analyses were conducted on all quantitative data used in this report for 
two purposes. First, analysis was done in order to document the presence or 
absence of differences between Transition, Comparison, and sometimes Attrition 
group children and families. The analyses conducted for this purpose include 
ANOVA (analysis of variance), repeated measures ANOVA, and ANCOVA (analysis 
of covariance). Second, multiple regression analyses were conducted to determine 
the importance of certain variables in outcomes for children. 

Summaries of the annual and longitudinal results and analyses are shown in Table 
2.1 and 2.2, respectively. Some of the longitudinal results from specific instruments 
are shown in other tables where differences exist or results are compelling. These 
tables are cross-referenced within Table 2.2. Additional statistical analyses conducted 
to determine important variables in outcomes for children are summarized in Table 2.10. 

All of the tables denote statistically significant results by "p < .05 or less." This 
indicates that the probability of obtaining the result by chance alone would be no 
more than five times in 100. If p < .01, the probability of obtaining the result by 
chance could occur one time in 100. The smaller the p value, the more confidence 
we can have that the result is not due to random chance. Actual significance levels 
attained are reported in each analysis. 




SUMMARY OF RESULTS FOR CHILDREN AND FAMILIES 
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Iiltfodaction 

An abundance of data has been collected throughout the life of the Transition 
Project, and results have been analyzed in a number of ways. This chapter presents 
results in three ways. First, a summary of findings from the annual reports is 
presented. These findings are catalogued in Table 2.1. Second, results are presented 
that look across the years of the study, providing the longitudinal analysis of data 
according to the original treatment group design. These findings are summarized in 
Table 2.2 and results from some instruments are displayed in other tables throughout 
the text. Finally, we present the additional analysis that were conducted to 
understand which factors are important in explaining results outside the treatment 
group design. The results from these analyses are presented in Table 2.1 0. Because of 
their sizes, these tables appear in Appendix C. 

Aimaal ReSaltS According 
to Treatment Groap 

After each year of the Transition Project, evaluators have presented the Arizona 
grantee with a report on the year's outcomes based on the national study's 
quasi-experimental design. The format created for the first annual report remained 
basically unchanged throughout the study. 

Most analyses of Transition Project data were conducted every year of the study. 

Some, however, were modified in order to optimize evaluation activities over time. In 
1 995, for example, case studies of the Transition schools were prepared in lieu of the 
standard format and analysis. Nevertheless, child outcome data were analyzed by 
treatment group that year. 

Some analyses also were modified because they had never shown minimal significance 
in early years and, therefore, research priorities were shifted to make better use of 
the growing body of evaluation findings. In addition, certain instruments were not 
analyzed consistently throughout the study because of changes in the types of scores 
the national research coordinating team made available to the state sites. For 
example, during the first two years of the study the national research coordinating 
team analyzed the Assessment Profile classroom observation instrument by 
percentage of "yes" scores. Different parameters, however, were used in later years 
by request of the instrument's authors. 
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Annual reports revealed several differences between treatment groups that are considered 
statistically significant. Only a few of these differences, however, remained consistent over 



described in the annual reports. Note that Table 2.1 includes only data up to Spring of 
1996 — a separate report for Spring 1997 would have focussed only on data for Cohort 2 
and was not considered an efficient use of time. 

The following sections present a summary of significant findings by outcome category 
from the annual reports. 

CHILD OUTCOMES 

Four significant differences were found between groups for the two math subtests of 
the Woodcock Johnson in 1 993 and 1 994. Three of the four differences favored the 
Transition group. In 1995 and 1996, differences on applied math problems appeared to 
favor the Comparison group, but those differences were nullified when children's 
primary language was taken into account. Primary language also erased significant 
differences on the Woodcock Johnson reading subscales in 1995 and 1996. 

Few group differences were found on items related to social/emotional and attitudinal 
measures. No differences were found on parents' perceptions of children's social 
skills. Children's adjustment to school did not differ except on two items in 1993 when 
Transition group parents reported their children getting along with the teacher and 
overall adjustment higher than Comparison group parents. 

Children's attitudes about school differed on only two items over the years. In 1994, 
Transition group children reported liking school more than Comparison group children, 
and in 1996, Transition group children rated their relationships with peers more 
positively than Attrition group children. 

One promising finding that emerged in 1994 is that Transition teachers rated their 
students more positively than did Comparison teachers on the Teacher Ratings of 
Students instrument. When researchers then examined teacher ratings of students 
relative to both actual academic achievement and observer ratings of classroom 
DAP they turned up some interesting patterns. 

High DAP teachers (DAP ratings of high, medium, or low based on Assessment Profile 
results) reported significantly higher perceptions of children's affective and academic 
growth than did medium or low DAP teachers. Furthermore, high DAP teachers had 
significantly higher perceptions of the importance of assertiveness in social situations 
and the frequency of children asserting themselves and exhibiting self-control than 



time. These can be found in Table 2.1, which summarizes the findings for Cohorts 1 and 2 



did medium or low DAP teachers. 










Nevertheless, students' Woodcock Johnson test scores in high DAP classrooms were consis- 
tently lower than found in either medium or low DAP classrooms, and students in medium 
DAP settings made greater gains than those in high or low DAP classes. 11 This finding 
reflects only the 1 994 data and was not duplicated in 1 995. 

Another Transition Project researcher associated with the local evaluation examined 
the relationship between parenting style (as measured by the Parenting Dimensions 
Inventory ) and children's Woodcock Johnson academic achievement scores. Total 
scores on the Parenting Dimensions Inventory completed when children were in 
kindergarten predicted about five percent of the variance in third grade Woodcock 
Johnson broad reading and math scores. Of the four Parenting Dimensions scales, 
parental responsiveness was most highly correlated with academic achievement. 12 

FAMILY OUTCOMES 

Few of the instruments used to track changes in family outcomes showed significant 
differences between Transition and Comparison groups in terms of use of social 
services, stability, self-sufficiency, parenting, and family involvement in children's 
education. In 1994, however, an analysis of the local family interview questions showed 
that Transition parents reported having significantly more opportunities at school to 
develop parenting skills and talk with teachers about their child. Furthermore, more 
Transition families were knowledgeable about local community resources. 

In 1 996, the instrument Family Involvement in Children's Learning indicated that 
Transition parents were invited to and attended more school-based activities and 
volunteer opportunities than parents at Comparison schools. That same year, a few 
small differences between groups were also found regarding family resources and 
public assistance program participation. This was the first year that such differences 
appeared. 



SYSTEM-RELATED OUTCOMES 

Perhaps the strongest results favoring Transition occurred on instruments that examined 
systemic change. The School Survey of Early Childhood Programs completed by teachers 
showed early on that Transition teachers rated their programs as higher in elements 
that constitute an early childhood orientation than did Comparison teachers. In 1994, 
however, fewer areas favored Transition, while one item favored Comparison. 
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The most consistent findings over time relate to classroom practice. The Assessment 
Profile classroom observation tool for DAP showed differences favoring Transition 
throughout the study on most elements of DAP. Table 3.4 in Chapter 3 clearly 
illustrates this pattern. (Due to changes in the way the instrument was analyzed after 
1994, only data from 1995 to 1997 are compared across time.) Another classroom 
observation instrument, ADAPT, was introduced to the study in 1995, and also showed 
significantly higher DAP ratings for Transition classrooms each year on every domain 
(see Table 3.5). 

Loilgitadiiial Aiialysis of Results 
According to Treatment Groap 

While annual data reports were informative to Project personnel and indicated possible 
trends, these reports were always considered "snapshots" of the Transition Project and 
their results were viewed as preliminary. To develop stronger conclusions about the 
Project, a comprehensive longitudinal analysis was undertaken drawing upon data collected 
from both cohorts during the full length of the Project. Table 2.2 summarizes these 
longitudinal findings regarding outcomes for children, families, system, and policy. 

The following section presents an analysis of longitudinal data for children and 
families. Findings related to systemic and policy change, however, will be presented in 
Chapter 3. There are a two primary reasons for this. First, the systemic change data 
cannot be analyzed longitudinally by treatment group because many of the 
respondents — teachers and family advocates — changed through the years as children 
moved up in grade; analyzing data from several respondents as if they were one would 
be inappropriate, but this data can be used to determine if a pattern emerges over 
time. Second, much of the data pertaining to systemic change was collected on 
locally-developed measures administered only to the Transition group, not to the 
Comparison group. Therefore, no direct contrasts can be made between groups. By 
relying on multiple data sources, however, inferences can be drawn. 

Other data that appear as shaded items on Table 2.2 will also be held for discussion in 
Chapter 3. These include certain questions on children and families that actually 
assess systemic factors related to the Transition Project. They are a part of a data 
set that is not technically longitudinal. 

QUESTIONS ABOUT CHILDREN 

All of the evaluation questions regarding outcomes for children hypothesized that the 
services provided through the Transition Project would result in greater gains by 
children in the Transition treatment condition than by those in the Comparison 
condition. The data, however, provide evidence that this hypothesis is not accurate. 









Relatively few differences exist between Transition and Comparison group children; 
those that occur are generally small and tend to favor the Comparison group as often 
as Transition. 

On cognitive measures, longitudinal analysis reveals no differences between groups in 
reading performance by third grade. Final math scores show the Comparison group 
outperforming the Transition group in third grade, while the reverse was true at 
kindergarten. One cognitive measure does favor the Transition group: vocabulary 
measures show the Transition group ahead of the Comparison group by third grade 
after starting out behind in kindergarten. The differences in both cases are small but 
significant. 

The locally developed Teacher Ratings of Students showed some differences 
throughout the years, but by the study's end in 1997, teachers rated Transition group 
children significantly higher in only one domain — logical/mathematical thinking. 

On measures of social and emotional development, Transition group children also do 
not exceed Comparison children. Longitudinal analysis of the parent version of the 
Social Skills Rating System, as well as the companion measure, Problem Behaviors show 
no differences between groups. The teacher version of the problem behaviors instrument, 
however, shows that third grade Transition students exhibited more externalizing problems 
(i.e., those outward directed problem behaviors such as aggressive behavior) than 
Comparison students. In fact, the overall standard score for behavior problems was 
significantly higher for the Transition group than those in the Comparison group, indicating 
a greater number of problem behaviors overall. 

Parent ratings of their child's health also showed no difference between Transition 
and Comparison groups. The only difference that was found among groups favored the 
Transition group over the Attrition group in frequency of dental visits. 

On the instrument that measured children's attitude toward school (What I Think of School), 
longitudinal analysis showed no differences between groups. Children across the study 
expressed uniformly high ratings. The scale, however, had a limited range (1 to 3), so group 
differences would be unlikely unless many students responded at the extremes. 

In parent ratings of their child's adjustment to school, significant differences appear 
on three of eight items. But each of these three items show the Transition group 
better adjusted to school than the Attrition group, not the Comparison group. This is 
shown in Table 2.3. One trend is worth noting: on six out of eight items, Transition 
parent ratings of their child's adjustment tend to improve from kindergarten to third 
grade, while Comparison group parent ratings decrease on six out of eight items, and 
Attrition group parent ratings decrease on all eight. While these differences are not 
statistically significant, they reveal a trend that may attain significance in the future. 











Table 2.3: Comparison of Treatment Groups Over Time on Your Child's Adjustment to School 



How much do you 
think your child 
likes school? 

How much effort do 
you think your child 
puts into trying to do 
well in school? 



TREATMENT 

GROUP 



TRANSITION 

COMPARISON 

TRANSITION 

COMPARISON 



SPRING 

KINDERGARTEN 



SPRING 
THIRD GRADE 



How well do you 
think your child 
actually does in 
school? 


TRANSITION 1 7.48 1 8.03* 

COMPARISON I J% I 783 


.016 


How well does your 
child get along with 
his/her teacher? 


TRANSITION 1 8.54 1 8.48* 

COMPARISON (136 8.44 


.034 


How well does your 
child get along with 
other children at 
school? 


TRANSITION | 7.55 ™ 797 

comparison 1 7.72 7.54 

■ • ; “'MITT T ■ 


... 


How pleased are you 
with the school's 
academic program? 


transition 7.84 7.91 

COMPARISON 8.51 7.98 

Hi BHWK IR&IHM 1 




How pleased are yau 
w/the school in terms of 
meeting your child's social 
& emotional needs? 


TRANSITION 7.9] ] 

COMPARISON 804 741 




How would you rate 
your child's overall 
adjustment to school 
at this time? 


TRANSITION 8.49 8.61* 

comparison 7.98 8.39 


.035 



Number of subjects: TRANSITION = 56; COMPARISON = 47; ATTRITION = 54 
Ratings are based on a scale from 0 to 10, where 10 is most positive 
* indicates that groups are significantly different 



QUESTIONS ABOUT FAMILIES 

Questions about families were developed under the hypothesis that Transition would positively 
impact families in a number of ways — from helping families obtain basic assistance, to improving 
self-sufficiency, parenting skills, and communications relationships, and involvement with the 
schools. A number of instruments were analyzed longitudinally to address these questions. 

Transition families were expected to receive more social service support through the school 
system and show more evidence of stability and self-sufficiency than Comparison families. The 
family background component of the Family Interview, however, revealed no consistent differ- 
ences between the groups over the course of the study regarding enrollment in social services. 
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The Final Interview for Families asked whether families had experienced any of 31 particular needs at any time during the Transition Project — also whether 
that need was met, and how. Table 2.4 shows the results. For the most part, Transition and Comparison groups had comparable needs for services through- 
out the project. Only four items showed significant differences. The Comparison group had greater need on two items related to children (after school care, 
and treatment for specific dental problems), while the Transition group had greater need on two items related to parents (substance abuse, counseling or 
drug treatment, and help with getting clothes, shoes, etc.). 

Only two significant differences appeared between groups when examining whether their needs were met. Both items favored the Transition group (help 
with getting clothes and shoes, etc., and transportation). 

The last component of the Final Interview for Families examined how families received needed services: whether they sought services on their own, were 
referred by someone from the school or the Transition Project, or whether they were referred by someone else. Results indicate that Transition families 
did receive services through the school significantly more often than the Comparison group (on 30 percent of the services), and many of the services 
were those targeted by family advocates. But such results must be interpreted with caution because only a small number of families expressed needs 
on any particular item. 



Table 2.4: Service Needs and Delivery by Treatment Group - Final Interview for Families 



Since your child entered 
kindergarten, has s/he needed: 


Percentage responding 
"yes" 


If yes, did you receive these 
services? 


If yes, received service 
through school/Transition 


After School Care 


T = 23.4% (n=l 5) 

C = 46.4% (n=26) p < .008 


T = 85.7% (n= 1 2) 
C = 69.2% (n=18) 


T = 50.0% (n=6) 

C = 11.1% (n=2) p < .05 


Dental Care 


T = 90.6% (n=58) 
C = 82.1% (n=46) 


T a 96.6% (n=56) 
C = 95.7% (rt=44) 


T = 46.4% (n=26) 

C = 1 8.2% (n=8) p < .02 


Treatment for specific dental problems 


T = 17.2% (n=64) 

C = 37.5% (n=21) p < .02 


T b 90.9% (n=10) 
C = 81.0% (n=17) 


T = 40.0% (n=4) 
C = 1 1 .8% (n=2) 


Counseling for emotional, behavioral, 
or adjustment problems 


T = 1 5.6% (n=10) 
C = 14.3% (n=8) 


T = 80.0% (n=8) 
C = 87.5% (n=7) 


T = 62.5% (n=5) 

C = 0 p < .04 


Free/Reduced Price Breakfast or 
Lunch Program 


T = 89.1% (n=57) 
C * 94.6% (n=53) 


T m 98.2% (n=53) 
C = 98.1% (n=52) 


T = 73.6% (n=39) 

C = 92.3% (n=48) p < .03 


Since your child entered 
kindergarten, have you or other 
adults in your home needed: 


Percentage responding 
"yes" 


If yes, did you receive these 
services? 


If yes, received service 
through school/Transition 


Unemployment Income 


T = 12.7% (n— 8) 
C = 8.9% (n=5) 


T = 87.5% (ft=7) 
C = 60.0% (n=3) 


T = 1 00% (n=7) 

C = 10.0% (n=3) p < .02 


College Training (2 or 4 year college/ 
university) 


T = 20.3% (n=13) 
C = 25.5% (n=14) 


T = 76.9% (n=10) 
C = 64.3% (n=9) 


T = 20.0% (n=2) 

C = 0 p < .02 


Parenting Classes 


T = 21.9% (n=l 4) 
C = 1 9.6% (n=l 1) 


T = 71.4% (n= 1 0) 
C = 80.0% (n=8) 


T = 1 00% (n=l 1) 

C = 50.0% (n=4) p < .04 


Health Care 


T = 62.5% (n=46) 
C = 64.3% (n=36) 


T = 97.5% (n=39) 
C=100% (n=36) 


T = 1 5.4% (n=6) 

C = 2.8% (n=l) p < .05 


Nutritional Services 


T = 23.4% (n=15) 
C = 20.8% (n=l 1) 


T = 80.0% (n=12) 
C = 100% (n=l 1) 


T = 58.3% (n=7) 

C = 9.1% (n=l) p < .05 


Substance abuse, counseling or 
treatment (drugs, alcohol, other 
substances) 


T = 10.9% (n=7) 

C = 0 p < .02 


T = 42.8% (n=3) 
C = 0 


T = 66.7% (n=2) 
C = 0 


Help with getting clothing, shoes, etc. 


T = 28. 1 % (n=18) 

C = 7.1% (n=4) p < .003 


T = 88.9% (n=16) 

C = 25.0% (n=l) p < .006 


T = 93.8% (n= 1 5) 

C = 0 p < .005 


Transportation 


T = 29.7% (n=19) 
C = 2 1 .4% (n=12) 


T = 94.4% (n=l 7) 

C = 66.7% (n=8) p < .05 


T = 1 7.7% (rt=3) 
C = 0 



Number of subjects: T=64; C=54 Table represents services in which significant differences between groups were found in at least one of the three columns. 

Survey included 1 1 possible child-related needs and 20 potential adult-related needs. Shaded cells represent findings 
that did not reach significance. ^ 
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Family self-sufficiency was addressed by questions regarding family use of public assistance and other items from the family background portion of the 
Family Interview. In all cases, families reported less use of public services by the spring of their children's third grade year, but no differences appeared 
between treatment groups. Families also reported higher incomes and greater employment, but again no difference appeared between treatment groups. 
Analysis of family risk factors similarly revealed no differences between groups. 

The Family Resource Scale also provided information about financial stability and self-sufficiency. Table 2.5 shows average ratings of Transition, 

Comparison, and Attrition group families on the adequacy of several areas of need. Parents' ratings of adequacy for the most basic needs, during their chil- 
dren's kindergarten year, are compared with their ratings at third grade. Comparison group parents rated resources as more adequate on two out of nine 
items during their children's kindergarten year. By grade three, one item favored Transition and one favored Comparison. Most striking is the fact that 
Transition and/or Comparison group parents rated resources as more adequate than Attrition group parents on seven of nine areas. Longitudinal analysis 
showed that the Transition group made more progress on adequacy of resources over time on one area and the Comparison group made more progress on 
another area. 



Table 2.5: Comparison of Treatment Groups Over Time on Family Resource Scale - Average Scores for 
Basic Resources 



RESOURCE 


TREATMENT 

GROUP 


SPRING 

KINDERGARTEN 


SPRING 
THIRD GRADE 


P ( 


Food for two meals 
per day 


TRANSITION 

COMPARISON 


4.3 

4.8’ 


4.8 

4.8 


'Tvs. C: .02 


Housing 


TRANSITION 

COMPARISON 


4.5 

4.7 


4.8' 

4.8' 


'T/C vs. A: .005 


Money to buy 
necessities 


TRANSITION 

COMPARISON 

sSlSMfi 


3.3 

3.6 


u — 
4.3' 


'T/C vs. A: .002 


Clothes 


TRANSITION 

COMPARISON 


3.7 

3.9 

j 


« 

4.4' 


1 C vs. A: .0001 


Heat for the home 


TRANSITION 

COMPARISON 


3.7 

4.4’ 

41- : ;v* •. 


4.4' 

4.8 13 

ai'.'fe 


1 T vs. C: .02 

2 Tvs. C: .05 

3 Tvs. A: .0003 

4 Repeoted Measure: .02 


Plumbing/water in the 
home 


TRANSITION 

COMPARISON 


4.4 

4.6 

11 44 


4.7 
4.9 1 


'C vs. A: .01 


Money to pay monthly 
bills 


TRANSITION 

COMPARISON 


3.6 

3.8 

11 


4.3 3 
4.7 


'Tvs. C: .05 
2 T vs. A: .0001 
3 Repeated Measure: .003 


Medical care for the 
family 


TRANSITION 

COMPARISON 


3.8 

4.1 

Wi:: 


4.1 

4.2 

;'!-r 


... 


Transportation 


TRANSITION 

COMPARISON 


3.7 

3.3 

'«y- 


4.2 1 
4.3 V 


'T/C vs. A: .03 



O Scale: Adequacy: 1 =Never; 2=$eldom; 3=$ometimes; 4=Usuolly; 5=Always. Number of Subjects: T=53; C=45; A=50 

■ ERIC L. 
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Certain Transition services such as parenting workshops and assistance from the family 
advocates were intended to improve parenting skills and bolster parent involvement in their 
child's education. Therefore, it was hypothesized, Transition parents would be rated higher 
on those behaviors than Comparison families. The Parenting Dimensions Inventory, however, 
showed no differences in parenting skills between groups. In addition, the Family 
Involvement in Children's Learning instrument showed no differences between groups in the 
types or frequency of communication and relationships with school staff, but Transition par- 
ents reported that their schools offered significantly more activities to parents than did 
Comparison or Attrition parents. Differences were also found favoring Transition over the 
Attrition group on the number of events parents participated in, and volunteer opportunities 
offered and participated in. (See Table 2.6). 



Table 2.6: Average Scores on Family Involvement in Children's Learning by Treatment Group - at Third grade 



Items w/significant differences 
between groups 


Transition 


Comparison 


Attrition 


P ( 


How many (out of 12) activities 
or events have been offered to 
you at your child's school? 


9.39 1 2 


7.54 1 


5.64 2 


’.001 

2 .0001 


How many of the activities or 
events offered did you attend? 


6.41 2 


5.35 


3.36 2 


2 .0001 


How many (out of 11) volunteer 
opportunities were available 
to you? 


6.73 2 


5.79 


3.98 2 


2 .0001 


How many of the opportunities 
did you participate in? 


4.05 2 


3.69 


2.23 2 


2 .0001 



Number of subjects: T=64; C=54; A=80 



Transition families were also expected to have higher participation in literacy and 
English as a Second Language programs because of support made available to them. 
The data, however, do not provide strong support for this hypothesis. Both groups 
showed similar participation in these activities: 1 S Transition parents versus 20 
Comparison families participated in GED training. Although these numbers are not 
significantly different, about 36 percent of those who did not have GEDs or high 
school diplomas at the beginning of this study took GED classes during the study. 
Neither did the Transition and Comparison group differ significantly in terms of 
participation in ESL classes. Overall, 1 8 Transition parents versus 1 1 Comparison 
parents took ESL classes. Taken together, this level of participation in ESL programs 
represents 45% of the families who identified themselves at the beginning of the 
study as speaking primarily Spanish in their home. 














po§§ible Explanations for Results 



Given the lack of strong findings each year of the Transition Project, it is not 
surprising to find few strong longitudinal differences by treatment group. Indeed, 
similar results have occurred at a majority of the other 30 state sites participating in 
the national Transition Project evaluation. Nevertheless, it is difficult to imagine why 
the Arizona Project — which overall, was highly implemented, staffed, and funded — did 
not generate more effects for children and families. A few cogent factors must be 
discussed that might explain the lack of strong effects. 

CONTAMINATION OF TREATMENT 

The first, and perhaps most influential factor, is the designation of a comparison 
group in a natural school setting. Simply defining a comparison group does not prevent 
its members from receiving some degree of treatment — and in this case, the potential 
for contamination was high. While the Transition treatment was intended to be limited 
to the children and families enrolled in Transition schools, each Comparison school also 
had a Head Start classroom on site. Throughout the study, from 1 5 to 34 percent of 
the Comparison group families had younger children attending Head Start classrooms. 
Consequently, each of these Comparison families received the assistance of a family 
advocate and other services provided by the same Head Start agency responsible for 
the Transition sites. 

In addition, Comparison schools and Transition schools do not operate in a vacuum: 
each school was simultaneously operating its own programs, state and federal grants, 
and district programs, many with goals that overlapped those of the Transition 
Project — such as improved parent involvement, higher student achievement, and 
increased access to comprehensive services for families. Appendix B presents a table 
of Transition-like practices and programs that were reported to be in place at Comparison 
schools in 1996. From this table, it is easy to imagine how the Comparison group could 
be contaminated. 

ATTRITION 

Unexpectedly high attrition rates also affected the study. (Attrition is defined as students 
and families moving out of their original treatment group.) Attrition rates not only 
proved to be considerably higher than anticipated at the national level, but they were also 
higher than average at the Arizona site. Table 2.7 below shows the large declines in total 
Transition and Comparison group children at each cycle of data collection. 
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High attrition poses a serious threat to the validity of a longitudinal study because it 
removes participants from treatment groups. Each year efforts were made by the 
evaluators to understand the makeup and dynamics of the Attrition group (by 
including in a number of analyses those in the Attrition group who could be located, 
tested, and interviewed). Further investigation has been undertaken for this report. 
Attrition data are analyzed and described under the section titled, "Attrition revisited." 
However, even after these efforts, it is still not possible to calculate how much the results 
would differ had these families stayed in their original treatment groups. 



Table 2.7: Numbers of Transition and Comparison Group Children in Study by Data Collection Cycle 





FALL 1 


SPRING 1 


SPRING 2 


SPRING 3 


SPRING 4 


% ATTRITION 


Cohort 1 


119 


108 


77 


57 


51 


57% 


Cohort 2 


128 


115 


95 


74 


68 


47% 



LANGUAGE 

Variations in primary language also may have skewed results. At the beginning of the 
study, almost one-third of children and families in the study spoke Spanish as their 
primary language. To accommodate these children, they were tested during the fall of 
their kindergarten year using the Spanish version of the PPVT, but a Spanish version 
of the Woodcock Johnson was never made available to the study. Therefore, 
Spanish-speaking students were not able to take this test until their teachers felt 
they were ready, and accuracy of teacher's judgements of children's readiness was 
likely to vary. Even after language minority children were taking the tests in English, 
language proficiency varied widely. While this factor was taken into account for 
analyses of student achievement, the degree of proficiency and its effects on test 
scores cannot be easily measured. 

PROBLEMS WITH MEASUREMENT 

A few of the instruments included in the national core data set may not have been 
appropriate for this study for a variety of reasons. The selection of a standardized 
achievement test was needed at the national level so that student scores could be 
combined and compared with other sites across the country. However, there is a 
potential problem in matching standardized tests to the program treatment. 
Proponents of DAP contend that such tests do not measure the types of skills 
emphasized in DAP classrooms, nor do they assess the motivation, self-confidence, 
creativity, and reasoning processes that the holistic method is designed to enhance. 
Philosophically, norm-referenced testing is considered inappropriate for assessing 
O DAP classrooms because DAP emphasizes individualized, child-centered education. 
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Some instruments had properties that made them problematic for other reasons. For 
example, the measure of children's attitude toward school, What I think of School, had a 
very limited range of responses (from one to three). This fact makes it unlikely to find large 
differences between groups because there is so little variation in scores. It was difficult for 
the national research coordinating team to locate appropriate instruments at the beginning 
of the study, so much so that some instruments were used differently than they had been 
intended. These instruments were developed for and validated on a different population 
than one similar to that of the Transition Project. For example, the Assessment Profile was 
originally developed for preschool classroom observations. The authors reworked the instru- 
ment to reflect primary grade classrooms, but a research version was used in the study. 

LIMITED DATA MEASURING FAMILY NEED 
AND PROVISION OF SERVICES 

Lack of data on the precise nature and extent of social services delivered to each 
family limited the ability of evaluators to identify program impacts in this regard. 

While it is clear that some families had greater needs than others, and that family 
advocates worked more intensely with certain families than others, no data detail 
how needs differed and how resources were allocated. It is unrealistic to assume that 
all families — whether in need of assistance or not, whether willing to participate in the 
program or not — had an equal probability of improvement, but data analyses were 
unable to separate out those who needed and received services from those who did not. 

STATISTICAL POWER 

Although the data from both cohorts were combined in order to increase statistical power, 
some analyses include data from only a small number of participants which makes the find- 
ings less reliable. Longitudinal analyses were mostly limited to children and families who 
had participated for the duration of the Project. Because attrition was so high, these num- 
bers were often on the low end of our ability to make valid comparisons. 

All of the above factors, operating alone or in combination, have the potential to 
obliterate consistent differences between treatment groups. But it is not possible to 
determine what effect, if any, they had. 

Ot&er Lotygitadiiial Analyses Cogdacted 

Because a number of confounding factors were identified as potential problems on the 
national core data set, local evaluators conducted additional analyses to determine 
whether previously untested factors might predict or explain some outcomes better 
than treatment group alone. This approach was also taken by other Transition Project 
site evaluations across the country. 












, U " 



ATTRITION REVISITED 



Evaluators and program managers suspected that the extent of measurable impact 
might be related to length of time that students and families were exposed to the 
Transition intervention. Therefore, they turned their attention to the Attrition group 
once again. Until final data were in, it had not been possible to assess the Attrition 
group's full effects — some earlier results even showed the Attrition group performing 
better than Transition group, which is inconsistent with research on student mobility. 

Multiple analyses were conducted to more fully understand the dynamics of the 
Attrition group. In longitudinal analysis, however, the Attrition group showed no 
differences from other groups either in repeated measures analysis or in differences 
between kindergarten scores and third grade scores. 

Evaluators then examined the distribution of Woodcock Johnson scores for the 
Attrition group and compared it to the distribution of Transition and Comparison 
group scores. It was suspected that there might be two types of students and 
families in the Attrition group: those who had attained greater financial stability and 
had relocated in more affluent areas, and those whose life circumstances were 
deteriorating. If this were true, the hypothesis was that some kind of bimodal 
distribution would appear in scores. This, however, was not the case. In fact, the 
Attrition group looked much like both the Transition and Comparison groups. 

To eliminate the possibility that the Attrition group's outcomes interacted with the 
original treatment group, evaluators analyzed the third grade Woodcock Johnson 
scores according to their original treatment groups. But no significant differences 
appeared between Attrition group members who started out in Transition versus 
those who started out in Comparison. 

Finally, evaluators looked at the length of time that Attrition group members were 
exposed to their original treatment condition in an attempt to uncover any 
differences due to time in treatment. In order to isolate the effects of time in 
treatment, the number of months each child and family spent in the Transition 
treatment condition were calculated. Those who spent the entire Project in the 
Comparison condition were given a time-in-treatment score of zero months. The rest 
of the participants were ranked two ways: some analyses used actual number of 
months in treatment, others used categories that were developed as follows: 



Time in Treatment Categories 

No treatment = 0 months in the Transition treatment (Comparison group) 

Minimal treatment = 1 to 9 months in Transition treatment 

Partial treatment = 1 0 months to 22 months in Transition treatment 

Maximum treatment = 23 to 36 months in Transition treatment 
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These analyses, however, also revealed no differences, as did the other analysis 
examining number of months in the Attrition group. In sum, the time-in-treatment 
variable did not predict any child or family outcomes. 

RISK FACTORS 

Research has shown that a variety of characteristics increase the probability of problems 
in school and life. One variable that had not been examined systematically prior to the lon- 
gitudinal analysis, was the degree to which a child and family was "at-risk" of negative 
outcomes. Drawing on the work of the national research coordinating team and other 
site evaluators, the local evaluation team put together an 1 1 -item risk index that 
would provide some indication of the factors working against successful child and 
family outcomes. The risks were identified as follows: 

Risk Factors 

• Parent has less than a high school diploma or GED (at first Family Interview) 

• Parent was less than 20 years old when child was born 

• Parent indicates chronic depression in 1 994 or 1 996 Family Interview 

• Family income was below the federal poverty level at first Family Interview 

• Family receives AFDC 

• Family receives SSI 

• Family has four or more children at home 

• Parent is the only adult in the home 

• Time and money subscales of the Family Resource Scale at final Family Interview 
are reported as inadequate 

• More than one move from child's kindergarten year through third grade 

• English not primary language at home at first Family Interview 

Averages were examined for each group, as well as for Attrition group members 
within their original treatment groups. Results are shown in Table 2.8 below. 



Table 2.8: Average Number of Risks According to Treatment 
Group at Beginning and End of the Project. 



TREATMENT GROUP 


AT BEGINNING’ 


AT END 2 


Transition 


5.53 


5.52 


Comparison 


5.55 


5.26 


Attrition 




5.74 




1 Number of subjects: T=1 1 8; C=88 

2 Number of subjects: T=66; C=54; A=86 
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No differences were significant. A separate analysis looking at the average number of 
risks by number of months in the Attrition group also revealed no significant differences. 

THE LANGUAGE FACTOR 

At the beginning of the Transition Project, approximately one-third of the children in 
the study spoke English as a second language at home. Of those, about 90 percent 
spoke Spanish as their first language. The fact that English was a second language for 
these children posed some concerns for the evaluation. First and foremost, a child's 
English proficiency affects scores on any tests administered in English. Specifically, 
such scores may reflect knowledge of English more than knowledge of subject matter. 

It is also widely acknowledged that most standardized tests contain an element of 
cultural bias. 13 

At the beginning of this study, teachers made a judgement as to which children would 
need to be tested in Spanish. Those children were tested in kindergarten with the 
Spanish 7V/P and the English equivalent, the PPVf. The Spanish version of Woodcock 
Johnson was never included in the national core data set due to the development of 
the test's norms which were based upon a Spanish-speaking population outside of the 
United States. Therefore, the Woodcock Johnson was not administered until the 
teacher felt the child had enough English proficiency to be tested on content 
knowledge and not English-language comprehension. At that time, the child took the 
English PPVT and Woodcock Johnson. 

For most of the Spanish-dominant children, testing in Spanish was discontinued after 
one year upon recommendation of each child's teacher. Even so, English proficiency 
varied greatly among students, with large gaps existing between conversational or 
reading English and proficiency at testing well in English. Test results were expected 
to vary greatly depending on primary language. 

To determine the part that language played in the achievement of students, the 
evaluators examined Woodcock Johnson and PPVT/IVIP scores both by the language 
the child was tested in during fall of kindergarten and by language used most 
frequently at home (these two were highly correlated), and then using language as a 
covariate. Surprisingly, language did not have as much of an impact on scores as one 
would expect from the literature. In fact, as Table 2.9 shows below, there was only 
one difference that proved to be significant. Non-English dominant speakers made 
greater gains from kindergarten to third grade in math scores. 
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Table 2.9: Comparison of Kindergarten and Third Grade Reading and Math Scores for English 
Dominant and Non-English Dominant Speakers 





English Dominant 
Speakers 


Non-English 
Dominant Speakers 


P< 


Kindergarten Spring 
Math RW Score 


423.1 


415.1 


— 


Grade 3 

Spring Math RW Score 


480.1 


481.4 


... 


Difference K to Grade 3 - 
Math Scores 


57.5 


65.8 


.02 


Kindergarten Spring 
Reading RW Score 


394.4 


392.3 


. 


Grade 3 Spring Reading 
RW Score 


474.2 


467.5 


— 


Difference K to Grade 3 - 
Reading Scores 


80.4 


80.6 


— 



Number of subjects for Kindergarten Woodcock Johnson (WJ) & difference scores: English Dominant: 1 32; 
Non-English: 22 Number of subjects for grade 3 WJ: English Dominant: 116; Non-English Dominant: 59 
Note: Number of non-English dominant subjects is higher at end becouse no Spanish version of the WJ was available, 
so the mojority were not tested with WJ in Spring of Kindergarten at all. 

RW=Rasch Wright 



The PPVT difference scores showed a pattern similar to the math difference scores. English- 
dominant speakers improved by ] 5.8 points from kindergarten to third grade, where non- 
English dominant speakers improved by 23.4 points (p < .0001 ). 

English proficiency also holds implications for family economic well-being. Job opportunities 
are limited for the non-English speaking population, and poverty is highly correlated with 
language minority status. Because of the possibility that primary language could be an 
important factor explaining family social and economic circumstances, each of the demo- 
graphic variables gathered during the family background portion of the first and last Family 
Interview was analyzed according to the primary language spoken by the family. None of 
the family variables, however, were dramatically different for the 32 percent of families 
who indicated they spoke a language other than English most often at home. Overall, the 
percentages indicated that the non-English speakers were slightly worse off economically at 
the beginning of the study than the English-speaking families but not dramatically so. 

The average number of risks per family did not differ by language either. English dominant 
families had an average of 5.9 risks, while non-English dominant families had an average 
of 5.8 risks. 
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Two other methods were used to analyze children's outcomes in hopes of revealing 
differences not already found by treatment group. Table 2.10 shows the range of 
these analyses. First the cognitive measures were reanalyzed with multiple 
regressions. Ten variables were included in the analysis to determine whether any 
could highlight or explain differences in outcomes between children. The variables are 
listed below: 

Variables Examined 

• Treatment group 

• Time in treatment 

• First language child was tested in kindergarten 

• Family involvement score 

• Inadequate Family Resources on nine most basic resources 

• Kindergarten Woodcock Johnson math Rasch Wright score 

• Kindergarten Woodcock Johnson reading Rasch Wright score 

• Family income at entry to project 

• School 

• Risk index score 

The only factors that predicted differences in cognitive outcomes were kindergarten 
reading and math scores. Kindergarten reading scores predicted 20 percent of the 
variance on third grade Woodcock Johnson math scores, while kindergarten math 
scores predicted six percent of the variance. Kindergarten Woodcock Johnson 
reading scores also predicted 33 percent of the variance on third grade reading 
scores, while kindergarten Woodcock Johnson math scores accounted for 25 percent 
of the variance on third grade PPVT scores. 

Data from the Social Skills Rating System were analyzed similarly. For parent 
perceptions of their child's social skills, number of risks for a family was found to be 
the most consistent predictor, though it only predicted a small portion of effects — 8 
percent of variance on the responsibility subscale and 7 percent on parent ratings of 
how cooperative they were. First language predicted 9 percent on the cooperation 
subscale. Number of risks also predicted 7 percent of outcomes for parent rating of 
behavior problems. 

Another method used to search for differences in child outcomes was to compare the 
highest and lowest performing twenty percent of third grade students on the 
Woodcock Johnson math and reading scores with the 10 variables. The hypothesis 
was that combining Transition and Comparison groups and examining only the very 
highest and lowest performers, key variables might be identified that could be likely 
factors in determining success or failure. 
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Interesting results emerged from this analysis (see Table 2.1 1 ). Top third grade performers 
on Woodcock Johnson math tests had significantly higher spring kindergarten math and 
reading Woodcock Johnson scores than did bottom third grade performers. Top third grade 
performers on Woodcock Johnson reading tests also had significantly higher scores for 
kindergarten reading, kindergarten math, and family involvement. 



Table 2.1 1: Variables that Differ for Highest and Lowest Performers at Third Grade 



Third Grade Math: 


Highest 20% 


Lowest 20% 


P< 


Kindergarten Spring 
Math RW Score 


425.8 


411.5 


.0001 


Kindergarten Spring 
Reading RW Score 


399.3 


386.5 


.002 


Third Grade Reading: 


Highest 20% 


Lowest 20% 


P< 


Family Involvement 


11.7 


9.2 


.03 


Kindergarten Math 
RW Score 


426.7 


412.6 


.002 


Kindergarten Reading 
RW Score 


401.3 


383.1 


.0001 



The highest and lowest 20% of scores in third grade represent o total of 48 students; RW=Rosch Wright 



This analysis was repeated using data from only the Transition group children, and while 
the number of subjects is too small to consider valid (top and bottom had 1 4 subjects each), 
the only difference that emerged was that the highest and lowest performers on third grade 
Woodcock Johnson math scores had significantly different family involvement scores and 
kindergarten Woodcock Johnson reading score did not differ. 

No variables were found to produce significant differences between highest and lowest 
performers on the Social Skills Hating System. The only variable to approach significance 
was the number of risks: the top 20 percent had 4.1 risks versus the bottom which had 5.2 
risks (p < .054). 










SUMMARY OF RESULTS REGARDING SYSTEM AND POLICY CHANGES 
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Many sources of data provide information about the Transition Project. Chapter 2 
presented the findings of the national core data set, which uncovered no strong evidence 
of differences accruing to the Transition group. While these findings may have been 
influenced by any or all of the confounding factors described in Chapter 2, the national 
core data set undoubtedly tells part of the Transition story. 

The local evaluation that has been carried on throughout the course of the Project tells 
another part of the story. Much of the local effort focused on systemic change using a 
variety of instruments and methods to understand the broad range of perspectives from 
parents, teachers, family advocates, principals, and other stakeholders. This chapter 
presents findings of the local evaluation regarding systemic change and specific 
Transition components. To provide another angle on the data, this chapter also includes a 
number of personal accounts of the Project's impact. 

Systemic Cfzange 

From the beginning of the Transition Project, it was anticipated that program services and activities 
would produce systemic changes both in the participating school systems and in the Head Start 
programs that feed them. Some of the expected changes would include improved methods for 
smoothing a child's transition from Head Start to public schools, changes in school policies to sup- 
port Transition-like educational and social services, implementation of DAP in classrooms, and 
better collaboration between staff of Head Start and the public schools. These changes were 
expected to become "institutionalized" and continue well beyond the end of the Project. 

To gauge the extent of systemic changes that occurred during the course of the Transition Project, 
evaluators gathered data from several sources. These included observations of activities in class- 
rooms, program meetings, and other functions; surveys of participants; focus group discussions 
with participants; and site visit reports organized by the national Transition research coordinating 
team. From these data, several evaluation questions can be addressed regarding systemic change. 

TRANSITION SERVICES 

Transition Project developers believed that Head Start staff, family advocates, Transition school 
staff, and social service providers would collaborate to provide more coordinated services to 
participants. This hypothesis was assessed in several ways as presented below. 
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A set of locally developed discussion items called the Local Family Interview Questions 
were developed to supplement the national core data set's Family Interview. The Local 
Questions were designed to gather information about essential services provided by the 
family advocates, such as home visits, goal setting, and referrals for health and social 
services. Table 3.1 summarizes parents' responses to the Local Questions. 



Table 3.1: Average Responses to Local Family Interview Questions 



Questions 


1993 


1994 


1996 


1997 


About how many times did your 
family advocate visit with you in 
your home? 


(Cl only) 


Cl: 3.2 


Cl: 1.5 


(C2 only) 


2.7 


C2: 2.4 


C2: 2.1 


1.7 


Did you talk with your family 
advocate about needs your family 
may have? 


66% yes 


Cl: 77% yes 


Cl: 54% yes 


51% yes 


C2: 71% yes 


C2: 62% yes 


Did you talk with your family 
advocate about setting goals for 
the year? 


31% yes 


Cl: 47% yes 


Cl: 58% yes 


38% yes 


C2: 38% yes 


C2: 52% yes 


Were you able to accomplish 


Not reported 


Cl: 81% yes 


Cl: 79% yes 


19% yes 


these goals? 


C2: 71% yes 


C2: 67% yes 


About how many times in the past 
year did your family advocate 


2.0 


Cl: 3.3 


Cl: 1.7 


1.3 


refer you to agencies for needed 
health or social services? 


C2: 2.0 


C2: 2.2 



Cl = Cohort 1; C2 = Cohort 2 



During the first year, participating families reported an average of almost three home 
visits, but for other years the average remained closer to two visits per family. Home 
visits by family advocates were considered central to the Transition Project. Discussing 
family needs with parents and attempting to set family goals were required activities of 
family advocates that usually occurred during the home visits. Throughout the study, a 
majority of parents recalled talking with the advocate about their needs, but the 
percentage of families who remembered setting goals was lower, ranging from 31 percent 
during the first year to a high of 58 percent in 1996. Of those parents who remembered 
setting goals, a clear majority (67 to 81 percent) described successfully completing their 
goals in 1 994 and 1 996, but only 1 9 percent said they were successful in 1 997. 

Family advocates were also expected to provide families with referrals to organizations 
and agencies for health or social service needs that families described. While the number 
of referrals varied widely for individual families, the average reported by all parents 
throughout the study ranged only between one and three. 














Transition services were also assessed by self-evaluation of family advocates and teachers at 
Transition schools. Each spring 1993-1997, advocates and Transition teachers completed 
checklists rating how they perceived the operation of transition services. Two different ver- 
sions of the Innovation Component Checklist were used. Family advocates rated transition 
services on a seven-item checklist, The Family Advocate's Role in Providing Transition 
Services; Transition teachers used an eight-item checklist, The Teacher's Role in Providing 
Transition Services. It is important to note that only some of the components on each check- 
list were similar in nature, and that ratings were not made by the same teachers and family 
advocates each year — staff turnover and the grade level progression of Transition students 
brought new staff into the Transition Project frequently. Furthermore, checklists were not 
used to assess Comparison school staff. Consequently, the results of the two checklists cannot 
be compared closely with each other, nor can they be analyzed longitudinally. Nevertheless, 
they offer some insights and illustrate possible trends. 



Table 3.2: Family Advocates' Average Ratings of Their Role in Transition Services - Innovation Component Checklist 



MEANS FOR 1 993 - 97 



YEAR 


COMP 1 


COMP 2 


COMP 3 


COMP 4 


COMPS 


COMP 6 


COMP 7 


OVERALL 


1993 


4.7 


4.0 


4.0 


4.2 


3.5 


3.5 


4.5 


4.1 


1994 


4.4 


4.3 


4.1 


4.4 


4.1 


4.5 


4.4 


4.3 


1995 


4.1 


3.7 


3.5 


4.5 


3.8 


4.0 


4.4 


4.0 


1996 


3.7 


3.8 


4.0 


4.2 


4.5 


4.6 


4.5 


4.2 


1997 


3.2 


3.3 


3.4 


4.2 


4.2 


3.9 


4.0 


3.7 


MEANS 


4.0 


3.8 


3.8 


4.3 


4.0 


4.1 


4.4 


4.1 



Component 1: Communicotion/colloborotion between family odvocotes ond teachers 
Component 2: Smooth transition of HS children from grode to grade, K-3 
Component 3: Parent involvement 
Component 4: Family services 



Component 5: Child health 

Component 6: Collaboration with special program ond social 
service staff 

Component 7: Documentation 



As shown in Table 3.2 and Table 3.3, overall ratings by each group stayed relatively high 
throughout the Project, though family advocate ratings (mean = 4.1 ) were slightly higher 
than teacher ratings (mean = 3.8). On those checklist items that were similar for both 
groups (i.e., communication between family advocates and teachers, parent involvement, 
and child health) the ratings tended to agree fairly closely. Both groups also “agreed" in 
that they gave lower ratings toward the end of the Project than they did other 
years — overall family advocate ratings dropped .5 of a point on the last year, while 
overall teacher ratings dropped .6 points over the last two years. 


























Table 3.3: Teachers' Average Ratings of Their Role in Transition Services - Innovation Component Checklist 



MEANS FOR 1993 - 97 | 



YEAR 


COMP 1 


COMP 2 


COMP 3 


COMP 4 


COMPS 


COMP 6 


COMP 7 


COMP 8 


OVERALL 


1993 


3.2 


4.6 


3.8 


3.2 


3.8 


4.6 


4.3 


4.7 


4.0 


1994 


3.4 


4.1 


3.3 


2.9 


4.2 


4.3 


4.1 


4.5 


3.9 


1995 


3.6 


4.5 


3.7 


3.5 


4.1 


4.3 


4.3 


3.8 


4.0 


1996 


3.5 


3.8 


3.1 


2.9 


3.9 


3.6 


3.8 


4.0 


3.6 


1997 


3.7 


3.3 


2.9 


3.0 


4.1 


3.5 


3.4 


3.6 


3.4 


MEANS 


3.5 


4.1 


3.4 


3.1 


4.0 


4.1 


4.0 


4.1 


3.8 



Component 1: Communication/ collaboration between teachers from grade to grade 
Component 2: Communication/collaboration between family advocates and teachers 
Component 3: Transition activities far students and families 
Component 4: Child educational services 



Component 5: Transfer of records 
Component 6: Parent services and involvement 
Component 7: Child Health 
Component 8: Social services 
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The steepest decline in final year ratings by both teachers and family advocates (.8 of a 
point below the overall mean for each group) occurred for an item that assessed 
collaboration between the two groups. That decline was supported by comments from 
family advocates and teachers indicating that they had experienced breakdowns in 
communication because of reduced staffing and changes in assignments that had 
increased workloads. Teachers also gave substantially lower final year ratings to other 
Transition services that involved family advocates: parent services and involvement, child 
health, and social services. Family advocates agreed with teachers in giving lower ratings 
to parent involvement the final year, but they did not give lower ratings to other family 
or social services. 

On the positive side during the last year of the Project, teachers gave higher-than-mean 
ratings to the communication and collaboration between teachers of different grade 
levels, and also to the transfer of records between grades — both considered important 
aspects of a smooth transition for students. Family advocates, on the other hand, gave 
lower final year ratings for the smooth transition of children from grade to grade. 

Focus group interviews with Transition teachers also addressed transition services. A 
number of teachers commented that Transition had boosted parent involvement overall 
during the Project, particularly by helping teachers communicate with Spanish-speaking 
parents. Some felt that an important contributor to increased parent involvement was 
the teacher home visit made possible by the presence of family advocates. Some 
teachers also noted that the Project had stimulated more communication between 
teachers of successive grade levels: others, however, felt that transition activities 
between grades were inconsistent at best. 
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Comments and observations about transition services were also made in annual site visit 
reports by the national Transition research coordinating team from 1993 to 1996. The 
reports noted a number of transition services that were evident in Transition schools, 
such as home visits and other family contacts by family advocates and teachers, 
communication between Head Start and kindergarten teachers, and inclusion of regular 
Head Start children in elementary school activities. Among the primary challenges that 
national evaluators noted was the high turnover among family advocates during the 
course of the Project. This turnover, the report said, would make it more difficult for 
the program to provide consistent, nurturing service to families. 

Transition services were also addressed by a committee composed of local Transition 
Project participants representing various roles (i.e., principal, family advocate, teacher, 
health manager, and parents). This committee completed the 1996 Program 
Implementation Profile (PIP), which was organized by the national Transition research 
coordinating team to document program implementation. PIP evaluators reported that 
several transition services were operating in the schools, specifically mentioning that 
transition plans existed for each child and family, systems were in place to transfer 
records from Head Start to kindergarten and beyond, and that teachers used 
transferred records to plan curriculum and instruction. 

DAP IN THE CLASSROOM 

Project developers anticipated that Transition school teachers would provide a 
developmental^ appropriate curriculum and receive staff development related to this 
teaching approach. The implementation of developmental^ appropriate practices in 
Transition and Comparison classrooms was assessed primarily by direct observations of 
trained observers. Each spring two different instruments were used. 

First, the observers administered the Assessment Profile of Early Childhood Progroms to 
rank classroom practice along six subscales. Results for the years 1995-1997 (see Table 
3.4) show Transition classrooms ranking higher than Comparison classrooms on almost all 
subscales. On two of the subscales (Learning Environment and Curriculum) the 
differences were significant (p <.01 ) across all three years. Transition classrooms also 
ranked significantly higher than Comparison classrooms on the Scheduling and 
Interacting subscales during the final year of the Project. 



O 



57 




n s* 










«**•>-* 



The second assessment of DAP by direct observation was made each spring 1995-1997 
using the instrument, A Developmental^ Appropriate Practice Template (ADAPT). This 
instrument is intended to provide a more holistic picture of classroom DAP by examining 
practices in three broad domains. Results for ADAPT (see Table 3.5) show that 
Transition classrooms earned consistently higher scores in all three domains every year 
1995-1997, and also received higher overall scores. All differences on ADAPT were 
considered significant (p <.01 ). 



Table 3.4: Average Ratings of Classroom Observations with the Assessment Profile 1995-1997 





1 995 


1 996 


1 997 


TYPE OF ASSISTANCE RECEIVED 


T 


C 


P < 


T 


C 


P < 


T 


C 


P < 


Availability of Learning Materials (9 items) 


6.6 


4.8 


.0080 


5.7 


4.3 


.0297 


5.8 


4.0 


... 


Learning Environment (19 items) 


13.3 


7.3 


.0001 


11.6 


6.9 


.0056 


11.5 


5.2 


.0002 


Scheduling (15 items) 


14.0 


13.4 


... 


12.4 


11.2 


.0281 


15.0 


12.0 


.0001 


Curriculum (28 items) 


24.0 


14.4 


.0001 


20.7 


15.2 


.0094 


23.0 


16.0 


.0001 


Interacting (23 items) 


21.0 


15.2 


.0001 


17.0 


14.4 


... 


20.0 


16.6 


.0430 


Individualizing (18 items) 


14.4 


12.7 


.0201 


13.0 


13.2 


... 


14.0 


14.2 


... 



Table 3.5: Average Scores for ADAPT 1995 - 1997 





1 995 


1 996 


1 997 


DOMAIN (30 points possible per domain) 


T 


C 


P < 


T 


C 


P < 


T 


C 


P < 


Curriculum and Instruction: Promoting 
Children's Academic Development 


20.1 


11.6 


.0001 


17.5 


12.5 


.0049 


18.9 


14.0 


.0032 


Interaction: Supporting Children's Social 
and Emotional Development 


21.9 


14.5 


.0001 


19.5 


14.6 


.0019 


18.9 


14.7 


.0042 


Classroom Management: Facilitating 
Children's Overall Development 


21.1 


11.3 


.0001 


18.4 


12.6 


.0022 


19.6 


14.1 


.0012 


TOTAL SCORE 


63.1 


37.4 


.0001 


55.4 


39.6 


.0018 


57.4 


42.8 


.0012 



DAP was also assessed by teacher self-evaluation. Each spring, Transition teachers (but not Comparison teachers) ranked their own classrooms and schools 
for developmentally appropriate practice using the instrument, Innovation Component Checklist for DAP (ICC-DAP). This checklist covers nine areas. Results 
for 1 993 to 1 997 (see Table 3.6) show that teachers generally saw their classrooms as ranking relatively high for DAP over the five-year span of the 
Project. Ratings showed little variation from year to year, despite the fact that different teachers were surveyed over time as students moved up from grade 
to grade and as staff turnover occurred. On average, most aspects of DAP were ranked at or near 4.0 on a 5-point scale (5.0 being highest) with the 
exception of "staffing." This item averaged almost a full point lower overall and trended sharply lower from 1993 to 1997 (from 3.6 to 2.8). The lower 
staffing scores were supported by written teacher comments indicating that student-teacher ratios were too high. 



O 

ERIC 



58 



40 

















Table 3.6: Teachers' Average Ratings of Their Use of DAP and Support for DAP - Innovation Component Checklist 



MEANS FOR 1 993 - 97 



YEAR 


COMP 1 


COMP 2 


COMP 3 


COMP 4 


COMPS 


COMP 6 


COMP 7 


COMP 8 


COMP 9 


OVERALL 


1993 


4.7 


4.7 


4.2 


4.1 


3.9 


4.4 


4.1 


4.4 


3.6 


4.2 


1994 


4.0 


3.9 


3.7 


4.2 


3.7 


3.8 


3.5 


3.5 


3.2 


3.7 


1995 


4.3 


4.2 


3.6 


4.3 


3.6 


3.8 


3.8 


3.9 


3.5 


3.9 


1996 


4.2 


4.2 


4.0 


4.1 


4.1 


3.5 


3.6 


4.0 


2.6 


3.8 


1997 


4.2 


4.4 


3.9 


4.2 


4.0 


3.7 


4.0 


4.2 


2.8 


3.9 


MEANS 


4.3 


4.3 


3.9 


4.2 


3.9 


3.8 


3.8 


4.0 


3.1 


3.9 



Component 1: Teocher - Child Interaction 
Component 2: Cultural ond Linguistic Integration 
Component 3: Curriculum 



Component 4: Instruction 
Component 5: Assessment 
Component 6: Environment and Materials 



Component 7: Parent-Teacher Relationships 
Component 8: Teacher Qualifications 
Component 9: Staffing 



During focus group interviews, Transition school principals and teachers commented on 
DAP in their schools. Principals generally stated that professional development sessions 
on DAP had been a key strength of the Transition Project, and that DAP methods would 
flourish in the classrooms. Teachers in focus groups applauded their professional 
development opportunities, saying that they had made them more child-centered in their 
classrooms. They noted, however, that the three Transition schools varied markedly in 
their commitment to the DAP philosophy, and that each school's degree of commitment 
affected how strongly DAP took hold in individual Transition classrooms. 

Site visit reports from the evaluation teams coordinated by the national research 
coordinating team also commented on evidence of DAP they found in Transition 
classrooms. In the annual reports of the national Transition research coordinating team 
from 1993 to 1996, evaluators noted observing a number of developmental^ appropriate 
practices in Transition classrooms, including small group instruction, the use of learning 
centers, classroom environments rich in language, and readily available fine arts 
materials. 

Evaluators completing the 1996 Program Implementation Profile (PIP) reported strong 
evidence of DAP, commenting that developmental practices had improved over time and 
that teacher training in DAP would have-long term impacts. The PIP report, however, 
recognized that implementation of DAP varied at the three Transition sites. These 
evaluators also predicted a diminishing of the effects of Transition as time, staff 
turnover, and loss of Project funding took their toll. 
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WORKING WITH AT-RISK STUDENTS Aima Berman 

AND CULTURAL DIVERSITY • Transition teacher 



Project developers expected that Transition teachers would become more skillful 
in working with at-risk children and that they would develop increased sensitivity 
to diverse cultures. Three instruments assessed this hypothesis. 

On the Assessment Profile instrument mentioned earlier, scores on the Learning 
Environment and Curriculum subscales favored Transition classrooms across all three 
years, while scores on the Scheduling and Interacting subscales favored Transition 
classrooms during the final year of the Project. All four items specifically included 
factors considered important when working with at-risk children or addressing cultural 
diversity. The Learning Environment subscale assessed whether the arrangement of 
classroom space encourages child independence and whether the classroom 
environment reflects the individuality of each child, while the Curriculum subscale 
assessed such things as the degree of attention to cultural differences, 
child-guided learning, and individualization of the curriculum. The Scheduling and 
Interacting subscales included factors such as the amount of time allotted to 
child-guided activities, the variety of activities scheduled, the amount of positive 
interactions with students, and the degree to which students are treated fairly and 
without bias. 

On the ADAPT - instrument mentioned earlier, All domains favored Transition 
classrooms. Several subscales within these domains assessed teacher skills with 
at-risk children and teacher sensitivity to diverse cultures. Among these were 
items that looked at support for students' social and emotional development, 
encouragement of cooperative learning and self-regulation among children, 
adaptation of instruction to student interests and prior knowledge, and integration 
of multicultural issues into the curriculum. 



B efore the Transition Project supported intensive 
professional development at her school, 
teacher Anna Berman had never considered "loop- 
ing" with her classes. In fact, she'd never heard of 
it. The practice involves keeping a teacher with 
the same group of kids through two or more 
grades, then having the teacher "loop" back to 
the lower grade to follow a new set of students. 
The purpose is to allow teachers more time with 
students so they can better understand the 
children's individual learning styles. 

Anna credits the Transition Project with stimu- 
lating her interest in this innovative technique. 
"Another teacher and I in the program decided 
to loop because we had been to a lot of confer- 
ences [about it] that were offered through the 
Transition Project... that and just being around 
Sandy [Foreman] and being in dialogue groups... 
and so we tried it.” 

Looping really starts to pay off during a teacher's 
second year with the same students. "It helps a 
lot being able to just start the year knowing 
exactly where the kids are and knowing where 
you need to take them... I feel like I saved 
three or four months, just started right in, I 
knew exactly where the kids were, what the 
next step should be... I didn't have to go 
through all that beginning of the year testing 
and wondering, and putting them in a group 
and seeing if it worked, then regrouping. 
Knowing [the students], I already knew how to 
motivate them individually." 

Looping — and the home visits promoted by the 
Transition Project — also helped Anna's relation- 
ship with her students' families. "That was 
probably the biggest thing about the Transition 
Project... having to do the home visits at the 
beginning of the year. That really set the stage 
for the year. It made [the students] more 
comfortable with me, and the parents also... 



It's like at the beginning we all built a relationship together and it just kept going throughout the year. And now [without Transition] 
that's lacking at the beginning and [the parents] come a couple of times throughout the year, but I don't think they ever feel as 
comfortable with us as they did when [Transition] was ongoing." 



The Transition Project’s emphasis on professional development changed Anna's classroom environment in other ways, too. For exam- 
ple, it introduced her to the concept of developmentally appropriate practice, or DAP. The effect has carried on after the end of the 
Project. "[DAP] is a term that is still in my mind, that I'm always challenged with.... I've definitely moved more toward having the chil- 
dren engage in what their doing, and to have some choice so they would be self-motivated — you know, choice in topics that they write 
about or research... [I’ve learned] that kids can be responsible to make choices and to direct some of their own learning." 












On the ICC-DAP instrument mentioned earlier, the three domains ranked highest by teachers 
overall from 1993-1997 (see Table 3.6) were Teacher-Child Interaction, Cultural and 
Linguistic Integration, and Instruction. All three addressed skills considered important for 
working with at-risk students or making the curriculum culturally sensitive, such as helping 
children develop self-esteem, social competence and autonomy; helping children resist biases 
and learn empathy for others, viewing diversity in the classroom as an opportunity for 
learning; and allowing children to select many of their own activities and work individually 
or in small groups most of the time. 

Collaboration 

Project developers anticipated that Head Start, in collaboration with the public Transition 
schools, would identify effective strategies for collaboration, and develop a model for 
creating and disseminating systemic change. 

To evaluate the Transition Project's success in improving collaboration amongparticipants, the 
Survey of Collaboration ms given each year 1995-1997 to all Transition school principals, 
family advocates, and Transition teachers. The survey consists of a total of 20 items under 
three headings: "Project Mission and Vision," "Project Development and Organization," and 
"Key Leaders." Responses follow a Likert-type scale (1 = strongly disagree; 5 = strongly 
agree.) Mean ratings were calculated for each group. Principals, teachers, and family advo- 
cates. Results are shown in Table 3.7. 



Table 3.7: Participant Group Average Responses to Survey of Collaboration 



AREA 


GROUP 


1995 


1996 


1997 




PRINCIPALS 


4.2 


4.0 


4.2 


Project mission and vision 


TEACHERS 


3.9 


3.2 


3.9 




FAMILY ADVOCATES 


if WM- 


, 


te M 


Project development 
and organization 


PRINCIPALS 
TEACHERS 
FAMILY ADVOCATES 


4.1 
3.7 

3.2 


4.2 

3.5 


4.2 
3.7 
> 3J 




PRINCIPALS 


4.5 


4.3 


4.3 


Key leaders 


TEACHERS 


3.9 


4.3 


3.8 




FAMILY ADVOCATES 


U 


4.6 


M 



Ratings for individual items ranged broadly, from 2.8 to 5.0, and varied considerably 
among groups. Principals gave the highest ratings overall every year, while family advocates 
and teachers alternated places in giving lowest scores. Principals also frequently rated indi- 
vidual items one point or more higher than teachers or family advocates rating that same 
item, indicating different perspectives on the success of collaboration. 

Overall, ratings remained consistent from year to year. For example, the average of all 
three groups' mean ratings was 3.8 in 1995 and 1996, and 3.9 in 1997. While these are 
not low ratings, they indicate that some areas of collaboration were not considered as suc- 
cessful as others, particularly among teachers and family advocates. f? * 













Among the collaboration items consistently rated highest (> 4.0) each year 
1995-1997 were those dealing with a) the opportunity for participants to discuss 
and have input on key issues; b) the effort made to consider cultural, racial, and 
ethnic differences; and c) the quality of the partnership among group members. 

Among the items consistently rated lowest (< 3.5) were those dealing with a) 
school commitment to the Project; and b) the mechanisms available to deal with 
barriers such as turf issues and denial. 

Table 3.7 shows Survey of Collaboration results according to the three subscales 
mentioned above. Average subscale ratings among the participant groups are 
very similar. However, the subscale dealing with key leaders is rated slightly 
higher. This domain covers issues related to the way that Project group members 
interact and communicate, consider cultural differences when planning transition 
strategies and activities, and whose organizational leaders are committed to the 
partnership. This subscale was the only one of the three to be rated at average 
at least a four over the three years the instrument was administered. 



Janice McGaim 

• Transition teacher 

K indergarten teacher Janice McGuirre remembers 
the Transition Project this way: "i never had so 
much professional development in 20 years of teach- 
ing." Professional development opportunities and the 
help of a family advocate were the defining features 
of the Transition Project for Janice. 

With previous experience as a pre-school teacher, 
Janice says she felt predisposed to embrace the 
DAP principles that Transition supports, and so her 
enthusiasm for learning more about DAP did not 
surprise her. What did surprise her was the way 
that other teachers, and even the school principal, 
changed as a result of the Transition Project 
Though some had little prior exposure to DAP, 
they soon started supplementing or entirely 
replacing their classroom approach with more 
developmentally appropriate practices. Now, 
Janice says, DAP has earned a permanent place in 
staff discussions of primary education. 



In the annual site visit reports of the national Transition research coordinating team 
from 1993 to 1996, evaluators noted that parent participation in the Project, and 
links to community service providers were major strengths of the Transition Project. 
Evaluators completing the 1996 Program Implementation Profile reported that most 
types of collaboration among partners occurred regularly. 

STRATEGIES FOR SUCCESSFUL COLLABORATION 

Case studies conducted on the Transition schools in 1995 provide some insight 
into the kinds of strategies that effectively support collaboration as well as those 
factors that might be considered barriers to successful collaboration. The effective 
strategies and barriers varied from school to school, with some factors considered 
to be strengths in one school and barriers at another. 



Janice's own classroom strategy was also greatly 
affected, in particular by a week-long conference 
with Lilian Katz, developer of the "Project 
Approach" for teaching young children. The Project 
Approach is a constructivist method that instead 
of building lessons around themes or units, relies 
on children as a primary resource in their own 
education, taking advantage of children's native 
interest in particular topics or problems in order 
to develop authentic research projects that 
actively engage them. As a result of the Katz con- 
ference, Janice and another teacher began con- 
ducting extended classroom projects in 
life science. 

While Janice feels that the professional develop- 
ment aspect of Transition was most beneficial to 
her personally, she gives the distinction of "single 
biggest benefit of Transition" to the presence of the 
family advocates. It was the family advocates who 
scheduled home visits for the teachers prior to the 
beginning of each school year and translated for 
Spanish -speaking parents during the visits. These 
home visits created a crucial link between parents, 
teachers, and the school, Janice says. The benefits 
of that link were paid out throughout the year, 
especially for Spanish -speaking parents. 



"Before [Transition], "Spanish-speaking parents would only come in if they had a problem," Janice recalls. But after meeting the teacher 
and working with the family advocate, these parents became comfortable with the school setting and began to see it as a positive envi- 
ronment. Because of this contact many parents expanded their notion of the role they could play in their child's education. 



But the effect of family advocates was even broader, says Janice, family advocates also worked with children in the classroom, helped par- 
ents advance their education and improve English language skills, and most importantly, helped the teachers in emergency situations. "I'd 
see a child with two rotten teeth or a blister in his mouth and I'd turn to the family advocate and she would get the child right to a den- 
tist." While the district can also provide emergency referrals, response time may be much slower, Janice points out. "There are five schools 
to serve {in the district] and the resources are just spread too thin." 

In retrospect, the Transition Project enhanced Janice's skills and experience as a teacher. It also connected her more closely with her stu- 
dents and their parents. She simply calls it "the best five years of my professional life." 
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Following are some strengths of the Project as found at one or more sites. While 
these strengths are not specifically strategies, they point to areas that promote 
collaboration. 



• Organizational/individual Support One school found support for the 
Project expressed at the district level. A district level program director sat on 
the Transition Advisory Committee for the first year of the Project and was 
knowledgeable about the Project's development and goals. Support of the 
teachers was also key to enhancing collaboration at one school. Their enthusi- 
asm for the Project's components made them valuable assets in advancing the 
goals of Transition and facilitating smooth collaborative efforts. 

• Matching philosophies or goals between school/district and 
Transition. One school s existing orientation and goals matched those of the 
Transition Project to a large degree. Staff had prior training in DAP and prior 
experience in providing families with comprehensive services. Because a com- 
mon language already existed with Transition, collaborative efforts to enhance 
comprehensive services and DAP started off at a higher level at this school. 

• Strong links with service provider* Two schools had strong partnerships 
with medical and family service providers. The Transition Project family advo- 
cates were able to draw upon these resources for families, thus freeing up time 
that would otherwise be spent trying to develop connections with community 
resources. 

• Strong prior experience with DAR. As mentioned earlier, one school had 
extensive prior experience with DAP. Consequently, the teaching staff was able 
to pursue additional training and extend their knowledge into more specific 
areas of interest to them. Because this school had little need for the Transition 
Project to concentrate on introducing the education component, more time 
could be devoted to other program components. 

BARRIERS TO SUCCESSFUL COLLABORATION 



and Raul 'Valenzuela 

• Transition family 

F or Jessie Valenzuela, the central focus of the 
Transition Project was the family advocate. 
"The family advocate," she says, “was always 
there for me when I had a question. The advo- 
cates were like friends." 

Jessie and her husband, Raul, have two children 
who participated in the Transition Project. Both 
children originally participated in Head Start 
also, while four other Valenzuela children were 
never involved in either program. 

Jessie recalls that the advocates assisted families 
in a number of ways. For example, they provid- 
ed information when requested and sometimes 
offered direct assistance, such as food boxes. 
The family advocate also helped Jessie's family 
enroll in AHCCCS (Arizona's version ol 
Medicaid), and the advocate was willing to make 
home visits when Jessie was not able to come to 
school. 

Jessie was particularly impressed by the active 
communication of her advocate. She might be in 
the classroom talking with the teacher aboui 
some problem related to her child's education, 
and she would discover the teacher already had 
a plan because she had been briefed earlier by 
the advocate. In turn, Jessie would often heat 
about classroom issues from the advocate. 



Just as some features facilitated collaboration at the Transition schools, other 
features limited the likelihood of success. 

• Lack of organizational/individual support for Transition Project 
philosophy. The family-centered service philosophy and DAP were too differ- 
ent for the teachers at principal at one school where traditional views of educa- 
tion dominated. While some teachers did change their views somewhat, a lack 
of fit between philosophies remained. 



Before the Project ended, Jessie's family had 
moved out of the Transition school area. She 
reports that her two Transition children are thriv- 
ing at their new school — both earning spots on 
the Principal's honor roll. Jessie, however, cred- 
its Head Start more than Transition for theii 
recent successes. 
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• Schaal/district philosophies not congruent with goals of Transition Project 

One district's policies toward early childhood education and assessment differed 
greatly from the goals of the Transition Project. For example, class sizes and a 
strict adherence to the existing assessment system in one district were inconsistent 
with DAP. Collaboration at the school level was not necessarily impeded because of 
this difference, but the potential for change was limited. Teachers understood this 
situation and it affected their attempts to modify their own classroom environments. 

• Limited parent involvement One school experienced very low parent involvement, 
especially among Spanish-speaking families. This inhibited communication and 
discouraged the teachers' and family advocates' efforts to collaborate. Through 
perseverance, however, the Transition Project staff increased participation and 

made strides toward collaboration. 

• High turnover among family advocates and the addition of new teachers: 

Each of the schools faced high turnover of family advocates from year to year. New 
teachers were also added each year as the Project progressed through the grades. 
Because collaboration is based upon good communication, as well as on building and 
maintaining relationships, turnover and the addition of new participants presented 
a considerable barrier to the Project. 

• Poor communication between participant group s: Communication between 
teachers and family advocates was listed by two of the three schools as a challenge. 

At one school, teachers were unfamiliar with the types of services that Transition 
could provide. High turnover of family advocates contributed to this situation and 
hampered collaborative efforts. 

• Isolation of the Transition Project from the schoof. At two schools, the 
Project's isolation from the rest of the school posed a barrier to spreading Transition 
practice. Teachers reported that other staff at the school viewed them as 
separate and did not know what Transition was about. Clearly, this was a major 
barrier. Attempts to address it were made, but it remained a challenge. 

Creating and Di§§eimiiatii2g a 
Model for Systemic Cfrarjge 

One program goal stated at the outset of Transition was to develop and disseminate a 
model for creating systemic change. This model would provide strategies for 
smoothing transition practices between Head Start and the public schools as well as 
from grade to grade. While the public schools proved to be a difficult system to 
penetrate, a number of practices were successfully introduced, and by the completion 
of the Transition Project the management team had developed two useful handbooks 
for making their learning experiences available to others. 
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Each handbook was designed to serve a different purpose and was oriented to a 
different audience. In combination, the two handbooks can bring both preschool staff 
and parents together for successful transitions. 

The first handbook is entitled, "Smooth Transitions: Making the Journey from 
Preschool to Kindergarten Together." It was developed as a guide for a practitioner 
audience consisting of Head Start program planners, Head Start directors, teachers, 
and staff of other preschool programs. It is written from the perspective of the 
preschool program and characterizes the transition process as a journey. 

The handbook is comprehensive. It names the types of partners that must be 
identified and worked with in order to create successful transitions; details the 
development of the Transition team and a Transition Action Plan. It provides 
examples of relevant documents; describes strategies that will increase the likelihood 
of success; and focuses on promoting continuity in a number of areas, such as 
classroom, family support services, parent involvement, records and information, and 
training and meetings of staff. It is also filled with suggestions about evaluating a 
program's progress. 

The handbook has application to a wide range of programs and has been distributed to 
a variety of groups and individuals including the superintendents of the school 
districts with whom SWHD works, principals in all the schools, kindergarten teachers 
who will receive SWHD's Head Start students, the regional Head Start program 
office, all the Head Start programs in Arizona, and those in the Western region. 

Future plans are to use the document for training family advocates and Head Start 
teachers. The Transition project manager (who was primary author of the handbook) 
also plans to use it in presentations for early childhood education organizations and 
other training projects. 

The second handbook is titled, "Southwest Head Start: Ready for School — A 
Handbook to Help Families Prepare for the Start of Kindergarten." This booklet 
serves as a parent guide for smoothing transitions from preschool into Kindergarten. 

It includes ideas for helping both adults and children before school starts, the first 
day of school, and during the school year, and it provides a checklist for families 
making a first school visit. The handbook is written in both English and Spanish and 
has been disseminated to parents of Head Start children at the year-end home visit 
and parent meeting. Future plans include disseminating these to teachers as well. 

The final collaboration goal called for identifying what a successful collaboration 
actually looks like. Determining whether this goal was accomplished is difficult. On the 
one hand, strategies have been highlighted that promote collaboration. On the other 
hand, data from the Survey of Collaboration reveals disparities between groups as to 
the level of collaboration. 













While the collaboration was rated successful at giving participants input and considering the 
needs of the population served, school district commitment to Transition practices was not 
viewed by many teachers and family advocates as genuine and was not enough to overcome 
some barriers, such as incongruence between Transition and school/district philosophy, a 
factor that has direct bearing on the adoption of new practice and policy change. 

In the 1994 annual report, local evaluators presented the analysis of Melaville and 
Blank regarding cooperative versus collaborative partnerships as a framework for 
viewing collaboration. 14 Their analysis revealed that the Transition Project was much 
more of a cooperative partnership than a truly collaborative one. Some of these 
elements have little to do with the intent or motivation of the groups involved but are 
based simply upon organizational features or limitations. For example, in collaborative 
partnerships, each partner can use funding leverage to advance collaborative goals, 
whereas in cooperative partnerships, the members network and share information. In 
the Transition Project, the public schools did not have authority to substantially 
change funding in favor of Transition goals. Head Start and the schools did, however, 
share information in their efforts to meet goals. 

Also, in collaborative partnerships, partners are able, politically or legally, to negotiate 
programmatic and policy changes and to make necessary budgetary and administrative 
changes, while in cooperative arrangements, the partners can advocate for, but are 
not empowered to negotiate policy changes. In the Transition Project, most public 
school staff and administration supported practices such as family service workers, but 
they lacked the authority to decide how these kinds of services would be continued in the 
future, nor did they have the authority to allocate funds for new priorities. 

After review of survey and focus group data, and hours of observation of the 
collaborative process, it is clear that, in as much as the Transition Project can 
accurately be described as a collaborative enterprise, it was carried out successfully. 

Overall, for most of its duration, it was more accurate to consider this Project as 
an example of cooperative partnership. However, toward the end of the Project 
certain school policies changed which made the partnership more collaborative 
and also promoted Transition-like goals. Most notably, districts began to give their 
schools more budget authority which enabled them to leverage funding and allocate 
future resources to meet common goals. In addition, a few of the schools and 
districts received competitive grants for specific programs such as dual language and 
the state early childhood block grant. This funding strengthened collaborative 
efforts as well. 
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Policy C&aiige 

The final evaluation question asked whether the results of the Project would 
affect state and local level public policies and fiscal support to reflect a more 
comprehensive plan for addressing child and family needs in a holistic manner. 

This does not appear to have taken place. 

While Transition Project management have made annual presentations to the school 
district partners, no transition policies have been revised at the district level. At the 
state level, also, no evidence suggests that Transition Project results have so far 
contributed to a shift in policy toward a more holistic view of families and children 
and the development of comprehensive plans for addressing child and family needs. 
However, Southwest Human Development staff are involved in a variety of child and 
family policy forums, such as the Phoenix Violence Prevention Initiative and the 
Arizona Head Start Collaboration which are attempting to influence policy decisions 
about children and families. To the extent that the agency is able to bring relevant 
lessons from the Head Start Transition Project to the discussion, the Project may 
make some contribution to local and state level policy change. 



Family Literacy 



Learning to Collaborate: 

• One School's Experience 

W hen the Transition Project ended in 1997, each 
school found itself with no family advocates, no 
additional monies for teacher training, and no com- 
prehensive program targeting disadvantaged fami- 
lies. But according to Carol, an administrator at one 
of the former Transition schools, they found them- 
selves with something else — a vision. 

"I would say the effects of Head Start Transition 
are more systemic than just a single program," 
Carol says. "I see its impact as influencing the 
mission of the school and its relationship to fami- 
lies. People here have become more aware that 
we always need to be thinking: How can we com- 
municate better with our families? How can we 
bring people into the discussion even if they don't 
see that as their role in a school?" 

Community Worker 

As an outgrowth of its new vision, the school solicit- 
ed a grant for a "community worker" with duties 
based partially on the role of family advocates. 
For example, the community worker helps wel- 
come parents to the classroom — much as family 
advocates did — by helping teachers develop par- 
ent activities. "Our experience with Transition has 
helped us in that regard," says Carol. 

In addition, the school's community worker and 
social worker undertook special training on how to 
develop leadership among parents in the local com- 
munity. "To even start thinking that way is really the 
influence of our work with Head Start and 
Transition," says Carol. "And there are other kinds of 
programs we are involved in today that we proba- 
bly wouldn't have started had we not been 
involved in Head Start and Transition." 



One example is the Family Literacy Program sponsored by Literacy Volunteers. Parents enrolled in Family Literacy attend ESL and GED held at 
the church next to the school in the morning, then they are scheduled as volunteers in their kids' classrooms during the week. The drawback 
for teachers is that the program requires coordinated planning time — time they have difficulty finding. But the Transition experience helped 
teachers be receptive. 



"When Literacy Volunteers came, a couple of the teachers said, 'This is like whot we had in the Transition Project and it really doesn't have tc 
be a lot of your planning time." And so two teachers talked the others into it based on their previous experience with a similar program in the 
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Transition Project." 
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Health Clinic 

Another example is the new health dink that operates one day a week at the school and is funded through 
the state tobacco tax. The clinic was borne out of a need to gel medical services for "notch kids" — those too 
poor for private medical insurance, but too well off for welfare. Operated by a nurse practitioner group affili- 
ated with ASU, the clinic is based on a Head Start model of basic physicals followed by periodic check-ups. "I 
see a pretty direct line/' says Carol. "Ten years ago we never would have thought to bring medical services 
into the school. Seeing Head Start and hearing them talk about how they use physicals and medical refer- 
rals to help families headed us in that direction. Seeing that you can identify kids who need medical ser- 
vices and deal with it in a manageable way is another fairly significant impact from Head Start and 
Transition." 

Welcome Center 

The school's welcome center also took shape under Transition Project influences, according to Carol. 

"Several schools around already have welcome centers or entry points, but some we looked at didn't 
seem too welcoming to us. Testing and registration has to be a part of it, but we were also concerned 
about welcoming families and establishing good communications with them." 

The concept was discussed at Transition meetings for awhile, and it even received some material sup- 
port from the Transition Project, but much more important, says Carol, was the Head Start model that 
the school had for dealing with parents. "A lot of what we know in that area we've learned from Head 
Start. We have always looked to them as a resource when it comes to communicating with families." 

The Collaborative Piece 

As the welcome center illustrates, the school staff do not always differentiate between the effects 
of Head Start and Transition. Both programs have been at the school for a number of years, and 
both have been operated by the same agency, and so have similar features. What does come 
through from the overall experience is a positive model for collaboration with an outside organization. 

"It is much easier for schools to run their own programs than to have other agencies come in because 
there are always tugs and pulls of power, and you either say goodbye to each other or you sit down and 
work it out," says Carol. "That takes a lot of people's time. But because of Transition, and seeing that 
when you work through those hard times you get some positive effects that pay off many times over, I 
think we have become more receptive to other agencies and groups that show interest in working with 
us." 

Such receptivity is crucial, says Carol, because working with "outsiders" will be the reality of the 
future for tightly budgeted public schools. "Given where we are with finances, we have to learn 
another way to work. We have to work collaboratively with a variety of other people. And that 
whole collaborative piece isn't talked about enough." 

"How do you learn to network with all these people and get everything going in positive ways?" 

She asks. "When I think of our school history, we did it first with Head Start and Transition." 
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DISCUSSION OF RESULTS AND POLICY IMPLICATIONS OF TRANSITION 




Iiltrodactioii 

Evaluation of the Transition Project involved five years of extensive data collection 
and analysis. As a result, we can now draw a number of conclusions regarding the 
effects of the Project on its intended beneficiaries. This chapter presents those 
conclusions. 

The chapter begins with a discussion of program results by the now-familiar 
evaluation categories of children, families, system, and public policy. Embedded in this 
discussion are lessons that can be drawn from the results, focussed questions for 
further study, and operational recommendations for future programs of this type. 
Afterward, the chapter addresses broader issues involving public programs that 
target families in poverty, and it concludes with public policy recommendations based 
on the knowledge gained from the Transition Project. 

Di§ca§§ioii of program 

As presented in earlier chapters, quantitative data provide few measurable 
differences between Transition and Comparison group children and families. 

Qualitative data, on the other hand, provide a picture of where and how systemic 
changes occurred, how collaborative partnerships worked, and what future attempts 
at collaboration can gain from this experience. The sections immediately following 
discuss results and recommendations by evaluation category. 

CHILDREN AND TRANSITION 

Student outcomes provide no convincing evidence that the Transition Project achieved 
its desired child goals in terms of cognitive, social, or emotional gains over Comparison 
schools, nor did the Project produce better student attitudes about school or student 
adjustments to school than Comparison schools. Both groups gained overall academic 
competency, but by third grade were losing ground compared to their age mates in the 
test's normative sample group. In spring testing during kindergarten, both groups' reading 
and math levels were about half a grade lower than the norm (i.e., at a level that the 
norming group scored near the beginning of kindergarten). By third grade, their 
scores were more than a full grade lower than the norm — comparable to the average 
child in the fifth month of second grade . 15 



O 

ERIC 



70 



5i 








Both Transition and Comparison groups, however expressed positive feelings about 
school and learning. How Head Start affected their test scores and their positive 
attitudes toward learning and their adjustment to school cannot be determined 
because the evaluation did not include a non-Head Start control group. 

The Transition Project component that most explicitly addressed children's education was 
the one emphasizing developmentally appropriate practices (DAP) in the classroom. 
Given that no research findings show DAP alone improves scores on standardized 
achievement tests, the hypothesis that Transition would produce gains may have been 
misguided. In fact, the evaluation found that students who had the highest 
achievement in 1994 also had teachers who used a combination of DAP and more 
traditional teaching methods. This suggests that the combination approach may be the 
best classroom practice, and it should be investigated in future research. 

DAP might seem more likely to produce gains in student attitudes and emotional 
adjustments to school, rather than direct educational gains, because it emphasizes 
the importance of the socio-emotional domain in the early years of children's 
education. Indeed, Transition children showed a positive trend on the Child's 
Adjustment to School instrument, while Comparison and Attrition group students did 
not. Perhaps given a longer time frame, some of these differences might reach 
significance. This would constitute a worthy follow-up study, but the evaluators 
recommend using a more appropriate measure of attitude if research were to 
continue. The response range of the instrument used in this study was considered to 
be too restricted to detect any differences that may have existed. 

Additional analyses of children's outcome measures were performed by local 
evaluators to determine whether any of the Project's central variables (e.g., time in 
treatment) were important predictors of outcomes. Results, however, primarily 
illustrated only the predictive quality of kindergarten test scores — particularly 
reading — on a variety of third grade test scores. Although this relationship is 
interesting, it is difficult to interpret. Does it point to the importance of 
kindergarten reading-readiness skills? Is it an artifact of IQ? Without a study to 
specifically examine this relationship, we can only speculate. 












FAMILIES AND TRANSITION 

Most families in both the Transition and Comparison groups improved their 
circumstances over time: they reported higher employment and income, and lower 
mobility and enrollment in assistance programs. Transition Project participants, 
however, did not show any significant advantage over Comparison group participants. 
For example, Transition families did not earn higher incomes than Comparison families, 
were not employed at higher rates, were not less likely to be enrolled in public 
assistance at the end of the study, and they did not acquire greater parenting skills. 
Other indicators for GED/ESL participation, school involvement, and communication 
also show no significant differences between groups. 

While it is a positive circumstance that all families became more self-sufficient, the 
lack of differences between treatment groups is disappointing. This is especially so 
considering that family services were based on the work of the family 
advocates — widely regarded as the cornerstone of the Transition Project. 

Nevertheless, it cannot be stated with certainty that the Project's family support 
services did not work. Many factors contribute to this ambiguity. 

First, the design of the national study may have diluted results. According to the 
national design, family advocates were required to provide services to an entire 
classroom of students and their families, even if only one Transition study child was in 
that classroom. Second, the study participants in the Transition classrooms may have 
had fewer needs than the other families simply because they were former Head Start 
families. In other words, previous Head Start services may have already resolved 
some family problems. Third, no documentation details the level of need of each 
family and the intensity of service provided by the Advocates, therefore no analysis 
of effects can be made. So, while anecdotal evidence demonstrates the benefit of 
Transition on some families, the statistical impact of these results may have washed 
out in the averages. Furthermore, the fact that 15-34 percent of Comparison group 
families also received direct Head Start services through other children in the family 
leads the evaluators to believe that a substantial amount of treatment group 
contamination took place, further complicating analysis. 













SERVICE DELIVERY QUESTIONS 

Specific delivery of services, hiring practices, and staff training fell outside the 
scope of this study because, by design, the evaluation was not intended to be 
process-oriented. Nevertheless, the annual examination of data related to systemic 
change naturally focussed upon the service-delivery process to some degree. 

Consequently, some questions arose related to service delivery and other 
process-related items, and these point to the need for further examination. The 
questions and discussion below offer a point of departure. 

Why didn't family advocates , with low case loads / have better results than 
similar providers at Comparison schools with much higher case loads? 

About half of the family advocates that SWHD hired for the Transition Project were degreed, 
which is a much higher percentage than advocates in regular Head Start. Nevertheless, a 
discussion of hiring practices is relevant here. Barbara Hanna Wasik, a professor of 
child development at the University of North Carolina, researched the effects of 
home visiting programs, and she notes that hiring paraprofessionals presents distinct 
advantages and disadvantages . 16 On the positive side, paraprofessionals may be 
familiar with local resources, and share important values and culture with program 
participants. This can increase trust in the program. On the other hand, 
paraprofessionals may not possess the clinical skills necessary to deal with serious 
personal problems that client families encounter. Therefore, paraprofessionals 
require intensive program support, training, and supervision. In the case of the 
Transition Project, a review of records and interviews with both teachers and family 
advocates shows that the advocates underutilized or were not uniformly 
knowledgeable about some of the services with which they linked their families, 
specifically, those relating to employability and employment. SWHD managers should 
revisit the agency's hiring, training, and supervising practices to determine whether 
they are sufficient to support their advocates' work. 

What were the expectations for family advocates , how was their success 
measured [ and how well did they respond to the requests of their clients 
and their supervisors? In recent years, federal, state, and local agencies have been 
increasingly held accountable for producing results from public funding. Consequently, ser- 
vice managers have had to rethink programs and adopt a more "outcome-oriented" model. 
This is no small task, but management expectations of staff are critical to making service 
programs succeed. In the case of the Transition Project, some parents and teachers 
noted a lack of responsiveness on the part of family advocates. Managers must 
consider how they defined an effective family advocate, what the expectations are 
for responding to clients and supervisors, and finally what the consequences should be 
for family advocates who did not prove effective. 
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What is the most efficient method for distributing limited resources — should 
you serve the families most in need or those most likely to succeed? 

Setting service priorities is one of the most challenging responsibilities for service managers. 
Yet it is clear that families vary in terms of their level of need, readiness to improve their 
circumstances, and ability to make changes happen. One way to prioritize may be to 
develop a profile that identifies families most likely lo benefit from services, and then 
concentrate services on those families. Certainly, such "creaming" may create some prob- 
lems. Nevertheless, in order to make the best use of available services, future program 
managers should attempt to determine which parents are most motivated to participate at 
a high level, and records should be kept of service provision and intensity to help uncover 
differences in results. This strategy is already being considered in another study commis- 
sioned by the Arizona Head Start Collaboration Project and SWHO. As part of this study, 
which will determine how Arizona Head Start agencies impact families, Morrison Institute 
researchers are helping develop indicators of family need, readiness, and service intensity 
so that results can reflect these central factors. 

What is the best way to maintain service continuity? Focus group interviews 
contributed much to the evaluators' understanding of service delivery through the Transition 
Project. Interviews with families and teachers, for example, revealed a common concern 
about continuity for family advocates. They questioned: Should advocates follow families as 
children move up through the grade levels, or should advocates stay with the same teachers 
from year to year? On the one hand, family relationships seem to benefit if an effective 
advocate stays with them throughout. On the other hand, collaboration on classroom 
services seems to benefit if the advocate stays with one teacher. The negative sides of 
both points were demonstrated each year of the Project when family advocate turnover 
and the addition of new teachers made new assignments necessary often breaking up 
prior relationships between teachers and advocates and with their families. Consequently, 
communications suffered, as did perceptions of service quality. 

One way to create effective continuity for advocates, teachers, and families may be 
the "looping" strategy pioneered by a few teachers and advocates in the Project. Both 
the teacher and the advocate stayed with the same set of students (and families) 
through two grade levels before looping back to an earlier grade and picking 
up a new set of students. This strategy should be tried with more teachers if 
coordinated services are truly to follow individual families. 

Other concerns also emerged from focus group 
interviews. Among the most often-repeated: 

• too much of the advocates' time was taken up by meetings or paperwork 

• turnover was too high among advocates and too much time was spent waiting 
for them to "get up to speed" 

• turnover among advocates hampered relationship-building with families 
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Of course some paperwork must be done in order to document progress, and meetings 
must occasionally be scheduled if joint planning is to occur. Some degree of staff 
turnover is also inevitable, especially in a project like Transition where its life span 
was limited, and people looked for work that paid substantially better than the family 
advocate salary. Yet these three items are management issues that must be 
continually addressed as programs try to strike the most efficient ratio between 
time spent on administrative necessities and time spent on direct service. No simple 
formula exists, but the goal should be to maximize staff contact with families. 

Evaluator observations also found that one of the greatest challenges for family 
advocates was a social service system that is fragmented at best, and nonexistent at 
worst. With families working through many difficult circumstances, one small 
glitch — such as a transportation failure — could mean a total loss of service. Why, for 
example, schedule a medical appointment for a family if the family members cannot 
get to the doctor's office? SWHD has shown tremendous initiative and creativity in 
addressing service delivery gaps. In addition to providing or securing transportation 
when possible, SWHD has developed networks with dentists, and secured service 
contracts to provide care for those children whose parents did not qualify for 
AHCCCS but could not afford dental coverage. Advocates also provided translating 
services during school events and in other instances where language barriers arose, 
such as during kindergarten registration, the first day of school, or in doctor's offices. 

SYSTEMIC CHANGE AND TRANSITION 

As an evaluation goal, systemic change presents some difficulties. How much change is 
needed and how long must it persist in order for it to qualify as "systemic"? Program 
goals that calls for systemic change in schools face additional difficulties because 
schools are dynamic enterprises in which both the workers and the 
products — students — undergo substantial evolution regardless of any special 
treatment or funded program. Treatment effects, therefore, can be completely 
masked by a school's natural and pervasive environment of change. Among the most 
influential factors of school change, both positive and negative, are the following: 

• normal development and progression of students from grade to grade 

• inevitable turnover among staff, particularly in less affluent districts 

• changes in administration and modification of school goals 

• vagaries in funding and the availability of programs 

• transformation of neighborhoods 

• fluctuation in economic trends 

• metamorphoses in educational beliefs and styles, including 
those based on theory or research 

• political changes at all levels — local school boards, the state department of 
education, the governor's office, the state legislature, and at the national level 



In spite of the many factors that affect the impact — or even visibility — of systemic 
change in schools, substantial evidence indicates the Transition Project produced 
some noticeable effect on both the public school system and the Head Start agency 
involved. While most of the data assessing systemic change were collected only at 
Transition schools, trends still can be gleaned from the data. 

In many cases, transition services became better coordinated for families. This 
happened when collaboration occurred among Advocates, public school teachers, Head 
Start teachers, and social service agencies. Coordination was most apparent when 
continuity was maintained among all the parties involved. 

The Transition Project's focus on DAP also created systemic impacts. Some of this 
result could be attributed to a self-selection process: teachers with a predisposition 
toward DAP may have intentionally gravitated to the Project, thereby creating a 
higher probability of success. Nevertheless, Transition teachers clearly gained a more 
thorough understanding of the DAP approach each year as a result of the Project, and 
they increased their DAP skills in the classroom. This occurred because the Transition 
Project produced a powerful model for carrying out an effective teacher training 
program. The Transition Project manager, who was well-versed in DAP herself, was 
largely responsible for this success. 

Some effective principles can be gleaned from the strategies 
the Project manager employed. For example, she consistently: 

• listened to and addressed teachers concerns about their classroom 
practice and followed up on them 

• provided current literature on DAP to teachers, whether solicited or unsolicited 

• enabled teachers to attend relevant conferences and workshops by providing 
funding for conference fees and substitute teachers to cover their classes 

• made time during meetings to discuss teacher reactions to readings and 
conferences, and to answer general questions 

• supported teachers' efforts to acquire more expertise, implement their current 
knowledge of DAP, and take risks 

The Transition Project also effected widespread changes in the way teachers work 
with at-risk students, and it increased their sensitivity to diverse cultures. The 
former change was partially accomplished by training in DAP Both changes, however, 
were largely expedited by the introduction into classrooms of bilingual and 
neighborhood-based advocates. Through the advocates' efforts to make teachers 
more comfortable with home visits and parents more comfortable in the classroom, an 
environment of mutual understanding between teachers and families developed. 
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This led, in some cases, to improved parent involvement. Increased sensitivity to cultures 
also spread school-wide due to the Transition Project. At the beginning of the Project, schools 
had few bilingual support staff. By Project-end, each school considered bilingualism as 
essential to meeting their student and family population's needs. As an example, school 
newsletters are now typically printed in both English and Spanish. 

What about the question of duration? Will any of these "systemic" impacts persist 
beyond the end of the Project? It is reasonable to assume that without advocates in 
the classrooms, home visits by teachers and other staff will cease, particularly for 
families that are monolingual Spanish-speaking. And though some schools already have 
a social service worker on staff with some responsibilities similar to those of an 
advocate, the case loads these staff members carry are many times higher than 
those of advocates, so the intensity of services to families will undoubtedly suffer, 
as will coordination of services in individual classrooms. 

In a larger sense, however, Transition appears to have created some lingering effects. 
Because of the collaborative and partnering experiences that Transition has provided, 
lines of communication have been opened between school staff and other agencies, and 
a willingness has been fostered among some staff to consider alternative solutions to 
enigmatic problems. These types of systemic effects are not as tangible as, say, 
referral frequencies or ratings of classroom practice, but they are clearly in keeping 
with the Transition model — and they may be more far-reaching in the long run. 

Systemic changes are likely to persist as long as a critical mass of school and 
collaborating agency staff continue to create positive experiences together by tackling 
educational and social problems as partners both inside and outside the schools. 

COLLABORATION 

Like systemic change, collaboration presents an ambiguous goal that is not easily 
defined or measured. Moreover, the concept is often unclear to those who are 
engaged in it. As mentioned earlier, the work of Melaville and Blank suggest that the 
Transition Project was technically more of a cooperative partnership than a 
collaboration. Nevertheless, a good deal was learned from the Project about working 
with other agencies toward a common goal. In addition, the Arizona Head Start 
Transition Project held an advantage over some other sites nationally because SWHD 
already had ongoing positive relationships with the Transition schools through existing 
Head Start programs at the schools. Still, much variation occurred among the three 
Transition schools. 



One of the primary factors that influenced success in working with other agencies 
was the readiness of the partners. Just as families may be ready (or not) to change, 
schools can also be characterized in this way. One school entered the Project clearly 
more ready than the others. This school was farther advanced with DAP training, had 
existing links with some social services, and had a philosophy consistent with 
Transition goals. While barriers to collaboration were not absent at this school, less 
time was needed to surmount the barriers and move forward. 

The other Transition schools, while less ready upon entering the Project, were not 
precluded from developing collaborative relationships. Their effort, however, required 
more communication and patience. From this experience, we might conclude that 
communication and patience, along with readiness, are the most important factors in 
creating a successful collaboration. 

Readiness also implies support, another critical factor in collaboration. The vignette titled, 
"One School's Experience," described in Chapter 3 underscores the importance of having 
staff support for collaborative efforts. Both individual and organizational support were 
needed to keep the efforts on track, particularly when barriers appeared. 

Several important barriers to collaboration occurred for Transition Project personnel. 

First among these was staff turnover among family advocates — a situation that 
creates the potential for breakdown in communication between groups. Added to this 
was the expansion of the Transition Project to new teachers every year, a factor that 
served to forestall significant buildup of momentum for Transition. Collaboration was 
further impeded by the knowledge that the Project would lose its funding at the end 
of five years. Given these factors, the Transition Project collaboration was probably 
as successful as it could be. 

Ultimately, both Southwest Human Development and the public schools involved 
became more aware of the benefits and pitfalls of collaborative endeavors as a result 
of the Transition Project. All developed an understanding of the time and patience 
required to be successful. From this experience, also, came three lessons that should 
be highlighted. Collaborative partners must: 

• retain key personnel and quickly train new personnel 

• possess staff and organizational support for the same goals and philosophies 

• use communication, patience, and experience to keep collaborative efforts on track 
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SWHD is currently applying its substantial experience in collaboration to a 
state-funded project that requires collaboration with other community-based service 
providers in developing a new child care center for low-income working families. 

Successful collaboration also appeared to produce an unexpected side effect: program 
blurring. When programs blended closely, as did Head Start and Head Start 
Transition, individual services and effects become indistinguishable to the 
beneficiaries. The story of Jessie and Raul Valenzuela's family illustrates that point, 
as did comments made in focus group interviews with some families. Instead of being 
viewed as a separate program, the Transition Project was viewed by some participants 
as merely an extension of their previous experience with Head Start. This should be 
considered a symptom of success. 

Effects of Traiisitioii 
or i Pablic Policies 

The final goal of the Transition Project was to impact state policy with the results of 
the Project. At this date, there is no evidence that this has occurred, but given how 
public policy is set, it may be unreasonable to expect one project's results to be 
sufficiently powerful to stimulate any changes in the short term. A better way to 
affect state policy may be through a coalition of organizations with similar goals. 
Transition Project personnel are currently assisting two such coalitions by applying 
the experiences they gained from the Project. 

One coalition is the Arizona Head Start Collaboration Project, which operates under 
the auspices of the Governor's Division for Children. Its members are drawn from all 
state and tribal Head Start agencies in Arizona. The aim of this partnership is to 
create an infrastructure of collaborative policy, planning, and service delivery in 
priority areas: health, child care, welfare, education, literacy, and activities related to 
children with disabilities. 17 SWHD personnel bring a great amount of experience to 
this organization armed with their developing research expertise, involvement with 
the regional Head Start training center, and lessons learned from their experience 
with demonstration projects such as Transition. 

The second such coalition is the Phoenix Violence Prevention Initiative (PVPI), which in 
early 1997 convened a wide range of organizations and civic leaders to develop 
Phoenix's first comprehensive violence prevention strategy. At the end of March 
1998, the PVPI steering committee had publicly issued support for four bills seeking 
legislative approval. 18 
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Three of these bills were comprehensive and family-centered in their approach: a domestic 
violence bill that creates stiffer penalties for domestic violence and provides notice to 
victims; a "Kids Care" bill that supports health care for children of the working poor; and an 
extension of the "Healthy Families" child abuse prevention program that identifies families 
at higher risk of abuse and intervenes through prevention and an intensive home visiting 
program. (SHWD operates the Healthy Families program in Maricopa County.) All three 
of these bills were passed during the summer of 1 998. 19 

PUBLIC POLICY IMPLICATIONS 

Recent research may provide an explanation for the Transition Project's paucity of 
measurable effects, and also suggest a direction for future programs. In 1 998, 
psychologists Sharon and Craig Ramey performed a meta-analysis of the 
effectiveness of early intervention programs for children and families. Their analysis 
identified a number of principles associated with strong and lasting changes in 
development. 20 Three of these principles apply directly to the Transition Project: 

• Program Breadth and Flexibility 

• Program Intensity 

• Direct Provision of Learning Experiences 

The design and implementation of the Transition Project showed strong consistency 
with the first of these principles — it had both breadth of services and a great deal of 
flexibility. The Project, however, was noticeably lacking in the other two areas. It did 
not provide intensive services to families, nor did it provide direct educational 
intervention and learning experiences to children. 

What is the threshold for program intensity according to the Rameys? Their review 
of research concluded that programs were more likely to produce measurable, 
longer-lasting benefits if they offered services more than one time per week to two 
generations of a family (e.g., parent and child). Clearly, the Transition Project's 
services were far below this benchmark — family advocates averaged only two home 
visits per family per year. 

Regarding educational experiences, the Rameys found that direct interventions for 
children produced larger and more enduring changes than peripheral interventions. 

The Transition Project design, however, relied heavily on peripheral interventions: its 
goal was to bolster the support system surrounding children through services to 
families and teachers. While this goal is worthy, and one that is supported by some 
research, direct intervention may be more effective. For example, the Johns Hopkins 
University program, Success for All, provides support services as part of the overall 
strategy, but its greatest strength is direct tutoring for children. 













Pablic policy Recop^eildatioiiS 



Any longitudinal study of a social program such as Transition naturally lends itself to 
consideration of public policy implications. Based on the results of the Head Start 
Transition Project and relevant research in this area, the evaluators make the 
following public policy recommendations: 

• RFP evaluation processes and standards should be reformed to favor pro- 
grams that provide intensive services. Many requests for proposals (RFPs) specify 
comprehensive services as a factor in awarding grants. Often, the winning proposal 
promises the greatest number of services, but the services promised must be spread 
thinly. A more effective RFP evaluation process would select programs that can 

deliver focussed and intensive services, so that they will provide the greatest 
potential for enduring effect. 

• Program designs should focus on attainable goals. Federal funds should support 
programs that set specific, attainable, and measurable goals. These goals — and how 
they will be measured — should be determined early in the design process, not inferred 
afterward from ambiguous proposal language. While some changes to original program 
designs are inevitable, particularly in continuous improvement models, a focus on mea- 
surable goals should keep program services on track and minimize modifications. 

• Accountability for results should be the cornerstone of new program designs 
and evaluation. The concept of accountability has been translated into federal law by 
the Government Performance and Results Act of 1993. Consequently, Federal agencies 
are now held responsible for producing program results rather than for merely comply- 
ing with regulations and measuring program "inputs." One positive outgrowth is that 
state and local governments — and even non-profits — have begun to create perfor- 
mance benchmarks and standards (e.g., state education standards for student competen- 
cies). If future funded programs are to show the effects they achieve, program develop- 
ers must consider accountability measures in their designs to ensure that program 
evaluations will capture results. 

• The value of large-scale demonstration projects must be weighed against 
the expense and associated threats to validity. Demonstration projects with a 
broad scope, such as the Transition Project, usually have funding costs that run corre- 
spondingly high. Yet they promise vast accumulations of data. Nevertheless, they can fall 
victim to difficulties that threaten their validity, such as widespread variations in services 
and data collection; cumbersome program modifications; and conflicts over regional val- 
ues and ethical concerns. Attempts to strike a balance between the need for regional 
variability and the need for standardization in research protocols and data collection 
often create a source of tension. But when data collection and program implementation 
vary greatly from site to site, the statistical methods designed to control such factors can 
be overwhelmed. 










Differing values and ethical conflicts among site directors in a large project can also 
affect research validity because resolution of these issues may compromise the 
study's practicality. In the Transition Project, ethical conflicts led to a decision to 
provide the services of a family advocate to every child in a classroom even if that 
classroom had only one study subject enrolled — a practice that diluted the services of 
the family advocates and minimized the probability that significant differences would 
appear between Transition and Comparison group families. With a smaller project, this 
compromise might have been avoided. 

• Future HHS reseorch should focus on determining the long-term effects of 
the Head Start Program, not solely on new demonstration projects. An April 
1 997 GAO study concludes that insufficient research on Head Start precludes wise pub- 
lic policy decisions on the program. The best use of HHS Head Start research money, 
therefore, would be to support a well-designed, high quality, longitudinal study — per- 
haps one similar to the Perry preschool research that followed Head Start children into 
their late 20s. This type of study is rarely funded because of expense and logistics, but 
the alternative is to speculate on short-term research that may have little generality to 
broader populations. 

• New progroms should align their gools with Welfare-to-Work reforms. 

As welfare reform carries forward, it will substantially affect the implementation of 
most new social service programs. Common sense dictates that new programs avoid 
conflicts with welfare reform. Future funding should, therefore, require that programs 
integrate their goals with the aims of welfare reform, either through direct job 
counseling and training services, or through collaboration with programs and 
agencies that provide such services. 
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...FROM TRANSITION PROJECT EVALUATION 

'• ***• * 



Evaluation Question 



Children 

1 . Do Head Start (HS) children in Transition classrooms maintain 
and/or show gains to a greater degree than HS children in 
Comparison classrooms on the following indicators: 

a) cognitive skills 

b) social and emotional development 

c) general health 

d) adjustment to school? 

2. Do HS children in Transition classrooms exhibit more positive attitudes 
toward school than HS children in Comparison classrooms? 

3. Do HS children in Transition classrooms experience a smoother 
transition and better continuity of programming from HS to 
kindergarten and from one primary grade to the next than HS 
children in Comparison classrooms?* 

Families 

4. Do Transition families receive more social service support through the 
public school system and show more evidence of stability and 
self-sufficiency than Comparison families? 

5. Do Transition families show better parenting skills and have more 
involvement in and support for education than Comparison families? 

6. Do parents in Transition schools participate in and complete more 
literacy and English as a Second Language classes and workshops than 
parents in Comparison schools? 



Finding 



a) differences on standardized measures for math favor 
Comparison; differences on vocabulary favor Transition; 
no significant differences on reading 

b) few differences favor Comparison 

c) no significant differences 

d) no significant differences 

no significant differences 

evidence of more early childhood programming and 
focus on continuity at Transition schools 



evidence of more services; no significant differences on 
stability or self-sufficiency 

no significant differences 

no significant differences 
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Samm^ty of findings... 



...FROM TRANSITION PROJECT EVALUATION 















Evaluation Question 



Finding 



7. Do parents in Transition schools perceive home-school 
communication to be more effective and satisfactory than 
parents in Comparison schools? 

System 

8. Do Transition schools provide a more coordinated service 
delivery system (i.e., continuous and comprehensive) than 
Comparison schools?* 

9. Do Transition schools provide a more developmental^ appropriate 
curriculum, more satisfactory communication strategies, better 
staff development, and more opportunities for parent participation 
than Comparison schools? 

10. Are Transition school primary level teachers more skilled in 
working with the special needs of at-risk children and families 
than Comparison school teachers?* 

1 1 . What does a successful collaborative process look like?* 



Policy 

1 2. Have the results of this Project affected state and local level 
policies and level of fiscal support that reflect a comprehensive 
plan for addressing child and family needs in a holistic manner? 



no significant differences 



differences favor Transition schools 



differences on Developmental^ Appropriate Practice 
favor Transition; no significant differences on 
communications; some differences on opportunities 
for parent participation favor Transition 

evidence of cultural and linguistic sensitivity in 
planning and implementation at Transition schools 



evidence of collaboration; barriers identified and 
collaborative strategies learned; positive experience 
promotes future collaboration 



not at completion of Project; however lessons 
learned from Transition Project will be applied to 
coalitions supporting child and family issues 
which may affect future policy 



*These questions rely on multiple data sources. Direct comparisons are not always possible as some data were not available for Comparison group schools. 




8 5 





Samm^ty °f Reco^p^eiidatioiig 






t « 



>w' 






INTERNAL POLICY (SERVICE DELIVERY) RECOMMENDATIONS 

SWHD managers should revisit hiring, training, and supervising practices to determine whether they are sufficient to support their advocates' work. Some 
advocates under-utilized or were not uniformly familiar with some services, specifically those related to employability and employment. 



Program managers should determine what is a working definition of an effective family advocate, what are expectations for advocates in responding to clients and 
supervisors, and what are the consequences for advocates who do not prove effective. These steps will help satisfy increased demands for program accountability. 



Future program managers should determine which parents are most likely to participate at a high level, and then keep records of service provision and intensity 
to uncover reasons behind results. This will help programs make the best use of available services. 



Program managers should continue to seek better service continuity for families and teachers. One positive strategy may be "looping," which keeps a teacher and 
advocate with the same set of students (and families) through more than one grade level before looping them back to an earlier grade to pick up a new set of students. 



Managers should reconsider how advocates allocate time. Although paperwork and meetings are essential, an effective balance must be struck between time 
spent on administrative tasks and time spent providing direct service. The goal should be to maximize staff contact with families. 



PUBLIC POLICY RECOMMENDATIONS 



RFP evaluation processes and standards should be reformed to favor programs that provide intensive services. 

Program designs should focus on attainable goals. 

Accountability for results should be the cornerstone of new program designs and evaluation. 

The value of large-scale demonstration projects must be weighed against the expense and associated threats to validity. 

Future HHS research should focus on determining the long-term effects of the Head Start Program, not solely on new demonstration projects. 
New programs should align their goals with Welfare-to-Work reforms. 
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Appendix A SUMMARY OF DATA COLLECTION INSTRUMENTS 









A Developmentally Appropriate Practice Template (ADAPT): This instrument was 
designed in 1995 by M. Gottlieb of the Illinois Transition Project evaluation to 
measure dimensions of teaching and learning associated with developmentally 
appropriate practices in early childhood classrooms. After an hour of observation, 
eighteen descriptors are each rated on a five-point continuum to assess three 
domains of classroom practice: curriculum and instruction, interaction, and 
classroom management. In addition specific criteria are used to complete an 
overall classroom summary. 

Assessment Profile for Early Childhood Programs: This classroom profile is 
intended to provide a quantitative measure of classrooms and teaching practices as 
they reflect developmentally appropriate practice. The Research Version used in 
this study includes 91 criteria organized into six scales: availability of learning 
materials, learning environment, scheduling, curriculum, interacting, and 
individualizing. Observation involves noting physical characteristics of the room 
and interactions between the teacher and the students. Each item is scored either 
"Yes" (observed) or "No" (not observed or not observed consistently). This 
instrument was developed in 1992 and modified in 1995 by M. Abbott-Shim & A. 

Sibley who were the Alabama Transition Project evaluators. 

Child Health Questionnaire: Teachers and parents rate the child's general health 
and well-being. The parent questionnaire was part of the family interview. 

Teachers also completed a similar questionnaire each year. 

Child Writing Samples: A writing sample from children during second and third 
grade was added to the national core data base in 1 995. A specific writing "prompt" 
was developed at the national level, and local test administrators gave each child 
the writing task during their annual spring testing. Writing samples were scored by 
the testing company, Measurement, Inc. Scores reflected student performance on 
the following five elements: Focus; support/elaboration; organization; 
conventions/mechanics; integration/global rating of effectiveness. 

Family Background and Update: This information is initially collected in the Fall 
family interview, with select items followed-up each Spring. Items relate to family 
demographics, socioeconomic factors, family characteristics, and support services 
the family received. 
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Family Involvement in Children's Learning: This instrument was developed at the 
national level in 1995 with input from the project site evaluators. Questions relate 
to school involvement and communications (school-family relationships) and family 
support for children's learning (in the home). 

Family Resource Scale (FRS): This instrument is designed to measure the extent 
to which different types of resources are adequate in households with young 
children. The scale includes 30 items rank-ordered from most-to-least basic. The 
FRS was developed by H. Leet & C. Dunst in 1985 and is based upon research on 
family systems theory which posits that needs drive behavior, and since these 
needs can be arranged in order from most to least important, family emphasis is 
likely to be directed to meeting unmet needs that are high in the hierarchy (i.e., 
the most basic). Parents respond using a five-point rating scale designating 1 ) not 
at all adequate, to 5) almost always adequate. 

Family Services End-of-Year Summary: This locally developed form, completed 
by Transition family advocates at the end of each year, includes a summary of the 
number of contacts made with Transition families, the number of referrals made, 
and the number of hours parents have documented for working with their child at 
home on school work and volunteering for or participating in school activities. 

Final Interview for Families: This instrument was developed at the national level 
to provide an indication of the level of family need for services, whether families 
received needed services, and where they obtained the service. This instrument 
was administered at the last family interview, and both Transition and Comparison 
families were asked about their need for and use of 30 services for children and adults. 

Focus Groups: Focus groups are small discussion groups designed to obtain 
information about the perspectives of various project participants and 
stakeholders toward the Transition Project. An interview protocol consisting of 
eight to ten open-ended questions is developed for each group, with the discussion 
loosely following the question outline. Participants are encouraged to engage in 
exchange of ideas and to explore various aspects of the project in depth. 

Innovations Component Checklist: Three separate checklists were developed 
according to the guidelines for constructing them provided by Hall and Hord 
(1987). They are part of the Concerns-Based Adoption Model for assessing the 
change process. Each instrument describes the components of an innovation, such 
as Transition services, and three levels of implementation, from no implementation 
to full implementation. Program participants (teachers and family advocates) then 
rate the degree to which they believe the program is being implemented. Teachers 
complete a checklist on developmentally appropriate practice and Transition 
services; Family Advocates complete a checklist rating Transition services. 
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Local Family Interview Questions: The local interview questions were designed to 
supplement the national family interview by asking questions directly related to 
transition services and local evaluation questions. Items included whether services 
were available to families, whether they were accessed, and whether they were 
adequate. One set of questions was asked of both Transition and Comparison families; 
another set of questions relating directly to Transition services was asked only of 
Transition families through 1994. After the development of Family Involvement in 
Children's Learning instrument in 1995, the more general questions were dropped and 
Transition service related questions were asked only of the Transition families. 

Parenting Dimensions Inventory: The PDI was developed by M. Slater and T. 
Power in 1987 and is based on a multi-dimensional model of parenting which 
integrates hierarchically organized concepts drawn from the parenting literature. 

The hierarchy attempts to account for the parent-child relationship on a 
continuum that ranges from abstract constructs of parenting through behaviors 
that characterize daily interactions. Four of the eight dimensions of the PDI are 
included in the family interview: nurturence, responsiveness to child input, 
nonrestrictive attitude, and consistency. 

Peabody Picture Vocabulary Test-Revised (PPVT-R): This test is designed to 
measure a person's receptive (i.e., hearing) vocabulary for Standard American 
English. It can be used with individuals ages 2 through 40 who understand 
Standard English to some degree. Each item consists of a choice of four pictures. 

The examiner names one and the child points to the picture named. Items are 
arranged in order of increasing difficulty and only the range of items appropriate 
to a person's abilities is given. 

Program Implementation Profile (PIP): The PIP was developed in 1996 by 
the National Head Start Public School Transition Demonstration Project directors for 
use at the local site level. Project directors developed this instrument to allow 
sites to rate themselves in terms of the degree of program implementation. 

Project directors believed this instrument was needed to augment the annual site 
visits and the implementation ratings developed by the national research 
coordinating team. In 1996, each of the 31 Transition Project sites completed 2 
PIPs: One completed by the Project director alone and one completed by a 
committee of Project participants from various roles in Transition which was 
consensus driven. 
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School Climate Survey: This instrument is designed to assess the quality and 
satisfaction of the school environment from teacher, student, and parent 
perspectives. In this study, it was given to the teachers and parents and adapted 
for principals as well. Published in 1987, the authors (E. Kelley, E. Howard, S. 

Miller, N. Schmitt, and J. Keefe) intended it to be used to foster school 
improvement. Nine of the instrument's subscales were used in this study: 
teacher-student relationships; security and maintenance; administration; student 
academic orientation; student behavioral values; student-peer relationships; parent & 
community relationships; instructional management; and, student activities. 

Social Skills Rating System: This system is designed to assess student social 
behavior. The SSRS provides a social skills rating based on norms from a national 
sample of over 4,000 children ages three through 18. The system is divided into 
subscales of cooperation, assertion, self-control, and responsibility. Parents and 
teachers rate individual children on how frequently they exhibit each of the 38 
social behaviors. They also rate how important those behaviors are to them as 
parents and teachers. This instrument was developed by F. Gresham, & S. Elliot 
in 1990. 

Survey of Collaboration: This locally developed instrument was designed to 
obtain individual perspectives about collaboration between the Transition partners. 
Completed by family advocates, teachers, and principals, it contains 20 items 
which are grouped into three categories: Project Mission and Vision (7 items); Project 
Development and Organization (7 items); and characteristics of Key Leaders (6 
items). A five point scale was used, where 1 =strongly disagree and 5=strongly 
agree. This survey was adapted from a survey designed by the Portland, Oregon 
Transition Project evaluators. Items are based upon partnership evaluation and 
collaboration concepts. 

Teacher Rating of Students: This rating system was developed by the evaluators 
of the Illinois Transition Project and adapted for use in the Arizona evaluation. 

The purpose of the measure is to obtain teacher judgments about student 
progress in eight areas of skill and adjustment to school. Ratings were made each 
Spring. The following eight areas were rated: self-esteem, cooperative learning, 
physical development, family support, interest in literacy, language development, 
comfort in school environment, and logical/scientific/mathematical thinking. 

Teachers rated each child on each area using a four-point scale: 1 ) requires 
considerable development; 2) needs some development; 3) generally a positive area; 
and 4) strongly positive area. 















Test de Vocabulario en Imagenes Peabody (TVIP): This test is the 
Spanish-language, Hispanic-American adaptation of the Peabody Picture Vocabulary 
Test-Revised (PPVT-R). It measures an individual's receptive, or hearing, vocabulary 
for single Spanish words and shows the extent of Spanish vocabulary acquisition. 
Test construction and administration are the same as the PPVT-R. 

What I Think of School: On this instrument, developed by M. Reid and S. 
Landesman in 1988, children report their own perceptions of their early school 
experiences through engaging in a dialogue with the examiner. Eight key questions 
are asked about the child's attitude toward school. Children indicate their 
responses by pointing to choices on a rating card. The three-point scale ranges 
from 1 ) least positive response, to 3) most positive response. The instrument con 
be used with children ages four to eight. 

Woodcock Johnson Tests of Achievement-Revised (WJ-R): This battery of 
tests measures cognitive abilities, scholastic aptitudes, and achievement. It can be 
used with individuals at all levels of education, preschool through adult. Four of the 
nine subtests included in the Standard Battery are used in the Transition study: 
WJ-22, letter-word identification; WJ-23, passage comprehension; WJ-24, 
calculation; and WJ-25, applied problems. 

Your Child's Adjustment to School: This instrument was included in the Family 
Interview. Developed by M. Reid and S. Landesman in 1988, this instrument asks 
parents to rate eight school-adjustment related items on a scale from zero to ten. 
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Comparison School Practice? arid Service? a? of 1996 



1 








ISrl^SB 


Balsz School 


Longview School 


Papago School 



Education 

Programs 


• Head Start preschool on site 

• 1 part-time school psychologist 

• 1 full-time counselor 

• Meet-The-Teacher Night each Fall 

• Carousel of Creativity 


• Head Start preschool on-site 

• 1 full-time school psychologist 

• 2 full-time counselors (1 is bilingual) 

• Early Childhood Coordinating 

• Committee works on transition activities 

• Kindergarten Round-Up 

• Family Night 

• School tours for Head Start children 

• Work Sampling portfolio 

• Exchange between Head Start 
ond kindergarten teachers 


• Head Start preschool on-site 

• At-risk preschool 

• 1 full-time school psychologist 

• 4-year state restructuring gront 
with focus on DAP 

• CLIP reading tutoring 

• Curriculum Night 

• Bi-monthly school newsletter 


Social 

Services 


• 1 social service worker, 1 day per week 

• Sexual assault education program 

• DES-sponsored before/after school care 

• ESL/GED classes twice weekly on campus 


• 1 full-time sociol worker 

• 1 ASU. social work intern (2 days/week) 

• Hispanic & Native American parent 
lioisons/ home visits 

• Notive American Center 

• ESL classes at school 

• Porent support groups 


• 1 full-time social worker 

• 1 community service worker 

• Numerous collaborative 

• Relationships with local agencies 

• ESL classes on campus 

• GED classes available through 
high school district 


Health 

Services 


• 1 full-time school nurse 

• Health Safari Van visits periodically 

• Referrals for glosses 

• Immunizations, other health services 


• School-based health clinic run by 

• St. Joseph's Hospital provides primary 
care to students who qualify 

• 1 full-time school nurse 

• 1 full-time health aide (LPN) 

• 1 nurse practitioner (daily visits) 

• Transportation and assistance 
with medical visits 


• 1 full-time school nurse 

• Referral sources for orthopedic 
services, glasses, and braces 

• Dentol screenings every other 
year through mobile dental unit 


Parent 

Involvement 


• Booster Club 

• Porent representatives on School 
Articulation Committee 

• Bi-annual Parent-Teacher Conferences 

• Reading is Fundomental program 
for parents and first grade children 


• PTA 

• Native American Parent Group 

• Bi-annual Parent-Teacher conferences 

• Monthly Kindergarten Family Nights 

• Planning site-based manogement team 


• PTA 

• Site-Based Management Team 

• Participatory Management Team 

• Bi-annual Parent-Teacher conferences 

• 2-year federal grant for increasing 
parent involvement 

• Homework Hotline 

• Weekly student academic 
checklist sent with each child 

• Parenting workshops on campus 

• Parent component of CLIP 
tutoring program 


Partnerships 


• Phoenix Police Officer on campus 
daily (Project D.A.R.E.) 

• Phoenix Fire Dept. Urban Safety 
Program weekly visits 

• Motorola partnership provides 
speakers and sponsors Whiz Kids 
reading program 


• Denny's, Sizzler award dinners 
for parent volunteers 

• West Side Foad Bank vouchers 

• Bar S Corporation partnership provides 
tutors, gives employment preference to 
Longview parents, pays rent on 

condo for 1 family 

• Parenting classes through 

• Parents Anonymous 


• Collaborative relationships with: 
Phoenix Fire Department 
Phoenix Police Department 
Project D.A.R.E. 

Jewish Family Center 
Job Search 
Friendly House 
American Cancer Society 
Pilot Parents 



• Gran) application in process for 
coordination of community services 
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TABLE 2.1: STATISTICALLY SIGNIFICANT DIFFERENCES FOUND FROM 1993 - 1996 



| INSTRUMENT 


SPRING 93 


SPRING 94 


SPRING 95 


SPRING 96 | 




C H 1 


L D OUTCOMES 




WJ-24 (calculation) 


Cohort: 1 # Favoring: T, p < .001 


Cohort: 1, Favoring: C, p< .01 (gain store) 


No differences 


No differences 


WJ-25 (applied problems) 


Cohort: 1, Favoring: T, p< .05 


Cohort: 2, Favoring: T, p < .05 


Cohort: 2 Differences initially appear 
to favar C, but na longer exist when 
covaried with language 


No differences 


WJ-22 (letter word recognition) 


No differences 


No differences 


Cohort: 1 Differences initially appear 
to favor C, but no longer exist when 
covaried with language 
Cohort: 2 Differences are due to lan- 
guage spoken (covariate effects) 


Cohort: 1 Differences are due to lan- 
guage spoken (covariate effects) 
Cohort: 2 Differences initially appear 
to favor C, but no longer exist when 
covaried with language 


WJ-23 (passage comprehension) 


No differences 


No differences 


Cohort: 2 Differences are due to lan- 
guage spoken (covariate effects) 


Cohort: 2 Differences initially appear 
to favor C, but no longer exist when 
covaried with language 


PPVT-R (vocabulary) 


Cohort: 1, Favoring: Transition, p < .05 


Cohort: 1, Favoring: T (English speaking 

students only), p < .05 

Cohort: 2, Favoring: T/C over A, p < .05 


Cohort: 2 A is significantly lower 
than T or C, p < .01 


Cohort: 1 Differences initially appear 
to favor C, but no longer exist when 
covaried with language 


What 1 Think of School 


No differences 


Cohort: 2, Favoring: T 

Item: How much do you like school? 

p < .05 


No differences 


Cohort: 1 Favoring: T over A on one 
item "How well do you get along with 
other children at school?" p < .05 


Your Child's Adjustment to School 


Cohort: 1, Favoring: T 

Items: 4. How well does your child get 

along with teacher? 

8. How would yau rate yaur child's 
overall adjustment to school? (p < .05) 


No differences 


No differences 


No differences 


Teacher Rating of Students 


No differences 


Cohort: 1, Favoring: T 

Items: Cooperative learning, literacy 

p < .01 

Items: Self-esteem, language, comfort 
in school, logical/mathematical think- 
ing, p < .05 
Cohort: 2, Favoring: T 
Item: Family involvement, p < .01 


No differences 


Cohort: 2, Favoring: T 

Items: Physical development, language, 

logical thinking, p < .05 




F A M l 


L Y O U T C O 


M E S 




Family Resource Scale 


Na differences 


No differences 


No family data analyzed by treatment 
group in 1995 


Cohart: 1, Favoring: C 

Item: Do you have adequate heat for 

house? p < .005 


Receiving Public Assistance 


No differences 


No differences 




Cohort: 1 & 2, Favoring: A 
Item: WIC nutrition program p < .05 
Cohort: 2, Favoring: A 
Item: Medical assistance p < .05 


Family Involvement in Children's 
Learning 


No differences 


No differences 




Cohort: 1 & 2, Favoring: T 
Item: Average number of school-based 
activities/events offered & average 
number attended; Average number of 
school-related volunteer opportunities 
p< .005 
Favoring: T 

Item: Rules for yaur child about TV 
viewing? p < .05 
Cohart: 2, Favoring: T 
Items: Average number of volunteer 
opportunities attended; Anything pre- 
venting you from participating in 
child's school? Has child's teacher 
suggested home activities to work on 
together? p < .05 



KEVCDir C = Comparison; A = Attrition; K = Kindergarten; p < = probability tbot tbe resub wos obtained by tbonce alone; Shading in column lor 1 995 designates that the instrument was not analyzed that year. 
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TABLE 2.1: STATISTICALLY SIGNIFICANT DIFFERENCES FOUND FROM 1993 - 1996 (CONT.) 



INSTRUMENT | SPRING 93 | SPRING 94 | SPRING 95 | SPRING 96 | 





CHILD 


0 


UTCOMES 


( c o n t . ) 




Local Family Interview 


Data not statistically analyzed 
in 1993 


Cohort: 2, Favoring: T 
Items: School given you chance to 
develop parenting skills? Have 
chance to talk with teacher about 
child's needs/qualities?, p < .01 
Items: Know where to go/who to ask 
about social, health, community 
services? Does school ask your 
opinion about school policy, decisions? 
(p< 05) 




Data not statistically analyzed 
in 1996 




S Y S 


T 


EM O U T C O 


M E S 




School Survey of Early Childhood 
programs (average ratings by teachers) 


Cohort: 1, teachers, Favoring: T 
Item: 11, Total Score p< .001 
Items: 1,4, 6, 9, 10, 12 p< .01 
Items: 2, 5, 8, p < .05 


Cohort: 1, teachers, Favoring: T 
Item: 5 p < .01 
Items: 6, 9, 10, 12 p< .05 
Favoring: C, Item: 7 p < .05 




No differences 


Assessment Profile (DAP in classrooms) 


Cohort: 1, Favoring: T 
Subscales: (1 out of 5) 
Interacting p < .01 


Data not made available to the local 
sites 


Favoring: T classrooms on five of six 
scales: Availability of learning 
materials; Learning environment; 
Curriculum; Interacting (All significant 
at p ( .01); Individualizing (p < .02) 
(see Table 3.4) 


Favoring: T classrooms 
Scales: Learning environment, 
Curriculum p < .001 
Scales: Availability of learning 
materials p < .05 


ADAPT 


Not administered 


Not administered 


Favoring: T classrooms on all three 
areas: Curriculum & instruction; 
Interaction; and Classroom manage- 
ment and total score p < .0001 (see 
Table 3.5) 


Favoring: T classrooms 
Scales: Curriculum & instruction, 
Interaction, Classroom management, 
total score p < .005 



KEY: T = Transition; C = Comparison; A = Attrition; K = Kindergarten; p < = probability that the result was obtained by chance alone; Shading in column (or 1 995 designates that the instrument was not analyzed that year. 
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TABLE 2.2: SUMMARY OF EVALUATION FINDINGS ACCORDING TO TREATMENT GROUP - LONGITUDINAL ANALYSIS 



EVALUATION QUESTION | INSTRUMENT AND ANALYSIS | FINDINGS | p < 



UESTIONS ABOUT CHILDREN 



Do Head Start (HS) children in Transition classrooms 
maintain and/or show gains to a greater degree than 
HS children in Comparison classrooms in cognitive skills? 


• WJ broad reading and math Rasch-Wright scares 
and PPVT standard scores from kindergarten to 
grade 3 

• Repeated Measures ANOVA 

• Difference scores - 3rd grade - Spring K math /read- 
ing, PPVT 

• Difference scores - 3rd grade - Spring K math/read- 
ing, PPVT - using Language as variable 


Reading Means - No differences 
Spring K: T=392.5 C=395.3 
Spring 3: T=474.0 C=477.9 
Math Means: 

Spring K: T=411.1 C=409.7 
Spring 3: T=478.2 C=483.7 
PPVT Means: 

Spring K: T=71.Q C=74.5 
Spring 3: T=97.3 C=96.7 

No differences 

Math difference by Language 

English: 57.5 Non-English: 65.8 

PPVT difference by Language 

English: 15.8 Non-English: 23.4 


.002 

.05 

.02 

.0001 


Do HS children in Transition classrooms maintain and/or 
show gains to a greater degree than HS children in 
Comparison classrooms in social skills and emotional 
development? 


• Family Interview, Social Skills Rating System & 

Problem Behaviors 

m Repeated Measures ANOVA - K thru grade 3 

• Teacher Questionnaire, Social Skills Rating System & . 
Problem Behaviors- grade 3 

• ANOVA 


Parent version: No differences 
Teacher version: 

Externalizing means: T=3.9 C=2.5 

Total Standard score for Behavior Problems: 
T=1 07.1 C=1 01 .3 


.03 

.04 1 


Do HS children in Transition classrooms maintain and/or 
show gains to a greater degree than HS children in 
Comparison classrooms in general health? 


• Family Interview, Child health questions: General 
health; medical home, insurance, hyperactive diagnosis, 
Rx to control behavior, speech problems, frequency 

of dental visits 

• Frequencies, Chi Square; Means, ANOVA - grade 3 


No differences for T vs C 
Difference between T vs A: 

Average frequency of dental visits: 

I=L4 (=1.5 A=1.9 

(scale: l=once/year; 2=every 2 years; 3=less) 


.02 


Do HS children in Transition classrooms maintain and/or 
show gains to a greater degree than HS children in 
Comparison classrooms in adjustment to school? 


• Family Interview, Your Child's Adjustment to School, 
and Additional Items 

• Repeated Measures ANOVA 


Transition and Attrition groups differ significantly an 3 
items (see Table 2.3) 


.05 


Do HS children in Transition classrooms exhibit more 
positive attitudes toward school than HS children in 
Comparison classrooms? 


• Child Instrument, What 1 Think of School 

• Repeated measures ANOVA from 1993 to 1996 


No differences - uniformly high ratings with little 
variance. 




Do HS children in Transition classrooms experience a 
smoother transition and better continuity of 
programming from HS to kindergarten and from one 
primary grade to the next than HS children in 
Comparison classrooms? 


• School Survey of Parly Childhood Programs - teacher, 
principals 

• Your Child's Adjustment to School 

• Focus group data 

• Observation of program activities, meetings, classrooms 

• Inference based upon triangulation of multiple data 
sources 


Transition teachers reported more continuity in program 
components than Comparison teachers. No differences in 
children's adjustment. 

Continuity and transition activities were improved through 
collaborative efforts among Head Start and public school 
staff. Parents indicated that family advocates provided a 
link between the school and the parents. 





Q U E 


S T 1 0 N S ABOUT 


FAMILIES 




Do Transition families receive more social service 


• Family Interview, Family Resource Scale 


Differences between T vs C on 1 of 9 resources from K-3; 


.04 


support through the public school system and show 


• Repeated Measures ANOVA 


Differences between T/C vs A on 7 of 9 resources at 




more evidence of stability and self-sufficiency than 


• Family Interview, Family Use of Public Assistance 


grade 3; (see Table 2.5) 


.01 


Comparison families? 


Programs . ANOVA 1993-1996 


No significant differences 


-- 




• Final Family Interview 


Differences between T vs C on 4 areas of need 






• Repeated Measures ANOVA 


Differences between T vs C on 2 needs met 


-.01 






Differences between T vs C on school/Transition providing 


-.05 






service on 9 areas of need (see Table 2.4) 






• Risk Factor Index . ANOVA 0 & 


No differences 


-- 



KE Y: Transition ; ^Comparison; A=Attrition; K=Kindergorten; Underlining designates the group fovored by the analysis; p < = probability that the result was obtained by chance alone; Shaded rows designate 
lout systematic issues, and dota are not technically longitudinal. 

r CONTINUED ON NEXT PAGE 
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TABLE 2.2: SUMMARY OF EVALUATION FINDINGS ACCORDING TO TREATMENT GROUP - 
LONGITUDINAL ANALYSIS (CONT.) 



EVALUATION QUESTION 


INSTRUMENT AND ANALYSIS 


FINDINGS j 


p < 


Q U E S T 1 0 


NS ABOUT FAM 


1 L 1 E S (coni.) 




Do Transition families show better parenting skills and 
have more involvement and support for education than 
Comparison families? 


• Family Interview, Parenting Dimensions Inventory 

• Repeated Measures ANOVA 

• Family Involvement in Children's Learning 

• Repeated measures ANOVA 


No differences 

Differences between T vs C on 1 item: 

Sum of activities school offered 
Differences between T vs A on 
Sum of activities attended 
Sum of volunteer opportunities offered 
Sum of opportunities participated in 


.001 

.0001 


Do parents in Transition schools participate in and 


• Family Interview, Family Background - enrollment 


No differences between groups for enrollment in literacy 


... 


complete more literacy and ESL classes and workshops 


in education programs 


or ESL programs 




than parents in Comparison schools? 


• Frequencies, Chi Square 






Do parents in Transition schools perceive home-school 


• Family Interview, School Climate Survey, 


No differences 





communication to be more effective and satisfactory 


Parents, teachers, principals 


No differences 




than parents in Comparison Schools? 


• Family Involvement in Children's Learning 








• ANOVA 







UESTIONS ABOUT SYSTEMIC CHANGE 



Do Transition schools provide a more coordinated service 
delivery system {i.e., continuous and comprehensive) 
than Comparison schools? 


• Final Family Interview • ANOVA 

• Innovation Components Checklist - Means 

• Focus group data 

• Observation / documentation of Transition team 
process 

• Inference made based upon triangulation of 
multiple data sources 


More Transition families received services through the 
school than Comparison families. 

Teachers and family advocates rated services to parents 
highly. 

Facus group participants described coordinating efforts 
and offering comprehensive services. The Project's efforts 
to make services continuous were viewed as more difficult 
to accomplish with changes in staffing and moving up 
through the grades. 




Compared to Comparison schools, do Transition schools 
provide: 

• a more developmental^ appropriate curriculum? 

• more satisfactory communication strategies with 
teachers and Head Start staff? 

• better staff development? 

• more opportunities for parent participation? 


• Assessment Profile • ANOVA: 1993- 1997 
•ADAPT • ANOVA: 1995 - 1997 

• Observation /documentation of Transition team process 

• Survey of Collaboration 

• Focus Groups 

• Family Involvement in Children's Learning • ANOVA 


Differences in DAP favoring T consistently (see Tables 3.4 
and 3.5). 

Survey of Collaboration and focus group data indicate 
that communication was improved. 

Focus group data support the benefits of staff development 
provided. 

Transition parents report more opportunities for involve- 
ment offered than Comparison parents. 


-01 

.001 


Compared to Comparison school teachers, are Transition 
school primary level teachers more skilled at working 
with the special needs af at-risk families? 


• Survey of Collaboration (item on cultural sensitivity) 

• Focus group data - Teachers, family advocates 
•ADAPT .ANOVA 

• Assessment Profile • AN OVA 

. Inference made based upon triangulation of multiple 
data sources 

OO 


Survey of collaboration shows Transition teachers, family 
advocates and principals believed the Project considered 
culture, race, and ethnic differences when planning 
strategies and activities. 

Focus group data supportive of Project staff's ability and 
concern in meeting client needs. Linguistic support was 
described as central to Project's success. 

ADAPT and Assessment Profile show evidence that 
Transition teachers individualized to meet children's 
needs and were more culturally sensitive than 
Comparison teachers. 





KEY: Transition; C=Comparison; A=Attrition; K=Kindergarten; Underlining designates the group favored by the analysis; p < = probability that the result was obtained by chance olone; Shaded rows designate 
questions about systemic issues, and data are not technically longitudinal. 





TABLE 2.2: SUMMARY OF EVALUATION FINDINGS ACCORDING TO TREATMENT GROUP - 
LONGITUDINAL ANALYSIS (CONT.) 



EVALUATION QUESTION 1 


1 INSTRUMENT AND ANALYSIS 1 


1 FINDINGS 1 


1 P * 


QUESTIONS / 


B 0 U T SYSTEMI 


C CHANGE ( c o 


n t . ) 



Based upon data gathered through case studies: 

• what are effective strategies and practices for 
collaboration? 

• is a model for change developed and disseminated 
to other Transition sites? 

• what does a successful collaboration look like? 

(i.e., was the Transition Project a successful 
collaboration?) 



1 Case studies 

* Observation/documentation of Transition team process 
' Surveys of Collaboration 

i Inference made based upon triangulation of multiple 
data sources 



evolution support the finding that SWHD and the schools 
learned barriers to collaboration as well as strategies to 
maintain relationships and retain the focus of the 
collaboration (see text, Chapters 3 and 4). 

A model for implementing a Transition program was 
developed and disseminated by SWHD. 

Surveys of Collaboration throughout Project were 
generally high (average 3.8 out of 5). Case studies, 
observation, and other local data provide evidence 
that, to the extent that it represented a true collaborative 
effort, the Transition Project collaboration was successful. 



QUESTION RELATED TO POLICY 



Did the results of this project affect state and local level 
policies and the level of fiscal support that reflect a 
comprehensive plan for addressing child and family 
needs in a holistic manner? 



Monitor state and local level policies related to early 
childhood education. 



No district policies were changed as a result of Transition. 

No state level policy changes directly attributable to 
Transition; However, SWHD personnel are regular 
participants in local and state level organizations, co- 
alitions, and other forums advocating policies that are 
comprehensive and positive for children and families. 
They now bring experience from the Transition Project 
to these discussions. 



KEY: Transition; (^Comparison; A=Attrition; K=Kindergarten; Underlining designates the group favored by the analysis; p < = probability that the result was obtained by chance alone; Shaded rows designate 
questions about systemic issues, and data are not technically longitudinal. 
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TABLE 2.10: ADDITIONAL ANALYSIS CONDUCTED WITHOUT REGARD TO TREATMENT GROUP 



QUESTION 


INSTRUMENT AND ANALYSIS 


FINDINGS 


p < 


What factors predict differences in third grade math 
scores? 


* Woodcock Johnson (WJ) broad math Rasch Wright 
scores 

• Multiple regression* 


• Spring Kindergarten broad reading score accounts for 
20% of variance 

• Spring Kindergarten broad math score accounts for 
6% of variance 


.0001 

.0001 


What factors predict differences in third grade reading 
scores? 


• WJ broad reading Rasch Wright scores 

• Multiple regression 


• Spring Kindergarten broad math score accounts for 
33% of variance. 


.0001 


What factors predict differences in third grade Peabody 
Picture Vocabulary Test scores? 


• Peabody Picture Vocabulary Test - Standard Scores 

• Multiple regression 


• Kindergarten broad math score accounts for 
25% of variance. 


.0001 


What factors predict differences in third grade social skills 
as rated by parents? 


* Family Interview, Social Skills Rating System & 
Problem Behaviors 

• Multiple regression 


Social Skills subscales: 

Cooperation: Number of risks predicts 7% and language 
first tested in predicts 9% of variance 
Responsibility; Number of risks Dredicts 8% of variance 
Assertion: No variables predict variance 
Self-control: No variables predict variance 

Problem Behaviors— 

Externalizinq: Number of risks predicts 7% of variance 
Internalizina: No variables predict variance 
Hyperactivity: No variables predict variance 


.0004 

.002 

.003 

.008 


What factors predict differences in third grade social skills 
as rated by teachers? 


• Teacher Questionnaire, Social Skills Rating System & 
Problem Behaviors 

• Multiple regression 


Social Skills subscales: 

Cooperation: K readina score predicts 13% of variance 
Assertion: No variables predict variance 
Self-control: No variables predict variance 

Problem Behaviors— 

Externalizinq: No variables predict variance 
Internalizina: K readina score predicts 11% of variance: 
Number of risks predicts 20% of variance 
Hyperactivity: Lanauaae first tested in predicts 1 1 % of 
variance 


.003 

.005 

.008 

.006 


Do children in Transition classrooms who score in the too 
and the bottom 20% at third grade differ on any 
variables in a systematic way? 


* WJ broad reading and math Rasch Wright scores 

• AN OVA - variables included those identified for 
multiple regression analysis 

^Number of subjects is small; 14 in each group. Shows 
a trend but cannot be considered reliable. 


On Math, grade 3: 

Family Involvement 
K Moth 

On Reading, grade 3: 
Family Involvement 
K Math 
K Reading 


TOP BOTTOM 

11.1 8.4 

431.8 414.6 

12.3 9.2 

427.7 414.5 

395.9 383.2 




.04 

.007 

.002 

.03 

.008 


Do all children, whether in T or C classrooms, who score 
in the top and the bottom 20% at third grade differ on 
any variables in a systematic way? 


* WJ broad reading and math Rasch Wright scores 

• ANOVA - variables included those identified for 
multiple regression analysis 


On Math, grade 3: 

K Math 
K Reading 

On Reading, grade 3: 
Family Involvement 
K Math 
K Reading 


TOP BOTTOM 

425.8 411.5 

399.3 386.5 

11.7 9.2 

426.7 412.6 

401.3 383.1 




.0001 

.002 

.03 

.002 

.0001 


Do the number of risks families have differ by treatment 
group? 


* Locally developed risk index** 

• ANOVA 


No differences 


... 


Do scores of Attrition group children differ by original 
treatment group or confound results in any way? 


* WJ broad reading and math Rasch Wright scores 

• Repeated Measures ANOVA 


No differences 


... 


Does primary language interact with children's 
achievement scores and confound the results of 
Transition / Comparison group analysis? 


* WJ broad reading and math Rasch Wright scores 
and PPVT scores 

• ANOVA 


Only difference favors non-English dominant speakers on 
difference from K to third grade math scores 
(See Table 2.9) 


.02 



KEY: Transition; Comparison; A=Attrition; K=Kindergarten; p < = probability that the result was obtained by chance alone 

* For alj multiple regressions shown, potential predictor variables included: Treatment group, time in treatment, first language, Family Involvement score, Family Resource Scale score, Kindergarten WJ Math 
and reading Rasch Wright score, family income at entry, school, Risk Index score 

**DjrL i 1( jex=[care giver education, teen parent, care giver depression, poverty, AFDC, SSI, 4 + children, 1 adult in home, Family Resource Scales (time, money subscales), frequent moves, ESL] 
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Policy (School of Public Affairs, 
College of Public Programs) 
is an Arizona State University 
resource for objective public 
policy analysis and expertise. 
Morrison Institute researches 
public policy issues, informs 
policy makers and residents, 
and advises leaders on choices 
and actions. 
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X 

Southwest Human Development 
(SWHD), a non-profit educational 
and human services organization, 
operates 36 federally funded 
Head Start classrooms in Central 
Phoenix. The agency also provides 
other comprehensive services to 
young children and families who 
face challenges related to health, 
abuse and neglect, mental health, 
poverty and disabilities. Today, 
SWHD is the largest community- 
based group of its type in Arizona. 
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